-- Logs begin at Fri 2022-07-15 14:39:30 UTC, end at Fri 2022-07-15 16:00:38 UTC. -- Jul 15 15:11:37.512647 dv-cinder-850015-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 15 15:11:37 2022] *** Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: nodename: dv-cinder-850015-1-iscsi Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: machine: x86_64 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: clock source: unix Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: pcre jit disabled Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: detected number of CPU cores: 4 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: current working directory: / Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: detected binary path: /usr/bin/uwsgi-core Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: your processes number limit is 47680 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: your memory page size is 4096 bytes Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: detected max file descriptor number: 2048 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: lock engine: pthread robust mutexes Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: thunder lock: enabled Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 15 15:11:37.527972 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 15 15:11:37.566276 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: Python main interpreter initialized at 0x55f0e4f4b9c0 Jul 15 15:11:37.566276 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: python threads support enabled Jul 15 15:11:37.566276 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: your server socket listen backlog is limited to 100 connections Jul 15 15:11:37.566276 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: your mercy for graceful operations on workers is 90 seconds Jul 15 15:11:37.566691 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: mapped 403077 bytes (393 KB) for 2 cores Jul 15 15:11:37.566691 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: *** Operational MODE: preforking *** Jul 15 15:11:37.566996 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: *** uWSGI is running in multiple interpreter mode *** Jul 15 15:11:37.566996 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: spawned uWSGI master process (pid: 80259) Jul 15 15:11:37.567213 dv-cinder-850015-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 15 15:11:37.569032 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: spawned uWSGI worker 1 (pid: 80269, cores: 1) Jul 15 15:11:37.569032 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: spawned uWSGI worker 2 (pid: 80270, cores: 1) Jul 15 15:11:37.569032 dv-cinder-850015-1-iscsi devstack@keystone.service[80259]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 15 15:11:42.946507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:11:42.946507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: from cryptography.utils import int_from_bytes Jul 15 15:11:42.947249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:11:42.947249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: from cryptography.utils import int_from_bytes Jul 15 15:11:42.947361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:11:42.947361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes Jul 15 15:11:42.947776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 15:11:42.947776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes Jul 15 15:11:42.996503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 15:11:42.996687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 15:11:42.996868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] command line args: [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 15:11:42.997148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 15:11:42.997281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ================================================================================ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 15:11:42.997570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] admin_token = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:42.997808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_dir = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:42.997974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:42.998635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] config_source = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:42.998635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] debug = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:42.998838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_config_append = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_dir = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.000908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.001161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.001280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.001475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.001731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.002065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 15:11:43.002254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 15:11:43.002799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] command line args: [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 15:11:43.003085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 15:11:43.003246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ================================================================================ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 15:11:43.003536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.003730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.003856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_dir = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.003934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_source = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] debug = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.004898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_config_append = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.005914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_dir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.006845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] publish_errors = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.007903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] standard_threads = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.008858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_journal = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_json = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_stderr = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] use_syslog = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.009854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.010043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.010270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.010353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.010425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] publish_errors = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.010507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.010799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.010892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.011040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.011111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.011219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.011302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.011467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.011540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.011705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.011787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.011943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] standard_threads = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.012036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.012178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.012265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.012407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.012497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.012577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.012657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.012750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_journal = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.012886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.012970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_json = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.013090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_stderr = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.013166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.013266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_syslog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.013367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.013499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 15:11:43.013608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.013842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.014958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.015986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.016221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.016452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.016628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.016715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.016906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.016989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.017987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.018940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.019933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.020977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.021955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.022949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.023915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.024882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.025949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.026917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.027894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.028862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.029988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.030954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.031958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.032921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.033874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.034843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.035980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.external = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.036969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.password = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] auth.token = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.037911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.038878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.external = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.039995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.040921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.password = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.token = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.041976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.042974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.043960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.044919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.045153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.045638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.045728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.045814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.045891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.045965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.046897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.047849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.048948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049025 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.049933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.050958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.051991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.052962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.053891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.054873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.055984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.056988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.057944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.058903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.059860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.060762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.061930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.062967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.063908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.064865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.065969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.066954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.067986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.068949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.069899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.070889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.071999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.072954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.073916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.074910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.075900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.076057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.076135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.076396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.076839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.076839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.076985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.077945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.078992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.079955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.080913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.081854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.082837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.083826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.084962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.085996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.086995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.087756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 15:11:43.087899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.088141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.088377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.088617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.088854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.089091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.089344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.089584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.089831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.090067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.090314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.090572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.090824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.091056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.091296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.091531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 15:11:43.091700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 15:11:43.119460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.119697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.119863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.122948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.123199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.123453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.128891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.132592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.159846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.160125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.160262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.160523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.305147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.305852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.354658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.354904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.355070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.355155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.420203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.420481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.420602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.420753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.420884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.421507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.446815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.446997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.447919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.448052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.448140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.459429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.459695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.472620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.472823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.472969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.473989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.474102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.474344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 15:11:43.491912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.492092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.492092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:11:43.492215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:11:43.492435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.492562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.492879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:11:43.493007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 15:11:43.493159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.493935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.505747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.506913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.514788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.515301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.518050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.518338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.518557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.518656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.518787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.518885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.519970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.520910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.521201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.547113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.547578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.548713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.548996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.549186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.549446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.554272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.554540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.554691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.554853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.554961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.555297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.562224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.562506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.562706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.562941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.566989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.567279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.567401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.567545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.567689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.567817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.567944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568185 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.568995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.569119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.569253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.587393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.587677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.587918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.588187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.592222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.593111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.595841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.596141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.596773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.597062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.601251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.602151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.605133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.605760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.608826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.609124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.609407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.609534 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.609675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.609837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.609963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.610094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.610284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.610416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.610713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.610998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.629555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.629834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.630100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.630370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.630619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.630894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.631158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.631421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.638237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.639469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.640021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.640273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.641179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.641427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.643461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.643757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.644034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.644324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.644601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.644728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.644871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.645186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.645483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.645764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.657946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.658226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.658503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.658807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.659087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.659359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.659499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.659630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.659768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.660047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.660309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.660585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.660874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.661145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.678817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.680288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.682056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.683542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.685424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.687042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.687167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.688643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.688925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.689207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.689483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.690408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.690711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.690989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.696773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.698190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.698316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.699841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.701738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.702030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.702314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.702610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.703222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.703508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.703790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.704067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.712095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.712381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.712664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.713258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.713545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.713822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.720666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.720947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.721224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.721499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.721622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 15:11:43.721762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.721972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.722942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.723930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.724055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.724184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.724312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.724434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.724661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.724896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.725191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 15:11:43.763010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f0e4f4b9c0 pid: 80269 (default app) Jul 15 15:11:43.763143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f0e4f4b9c0 pid: 80270 (default app) Jul 15 15:11:43.766764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a67a36-2233-4007-b2d6-690a72769804 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:43.767065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a67a36-2233-4007-b2d6-690a72769804 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:43.767345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a67a36-2233-4007-b2d6-690a72769804 None None] PATH_INFO: `/v3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:43.771644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Fri Jul 15 15:11:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:11:51.431897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:51.432514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:51.432514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:51.437132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:11:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:11:51.445056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:51.445510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:51.445602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:51.514793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: INFO dbcounter [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] Registered counter for database keystone Jul 15 15:11:51.539553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG oslo_db.sqlalchemy.engines [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80269) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 15:11:51.551855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writer thread running {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 15:11:51.736610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80269) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 15:11:51.737021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80269) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 15:11:51.792242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:11:51.793895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:11:51.923677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:11:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 15 15:11:52.110381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:52.189558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: INFO dbcounter [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] Registered counter for database keystone Jul 15 15:11:52.211863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG oslo_db.sqlalchemy.engines [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80270) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 15:11:52.223834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writer thread running {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 15:11:52.350346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811c4fb-3491-4514-928e-b661dfd47e3a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:52.355848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:52.356618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:52.356899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:52.357196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:52.366478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:52.419945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 15:11:52.420278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc33e3-cc61-466a-a832-9c85c116e04d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:52.431034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Jul 15 15:11:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:11:52.438207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:52.466245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d205b-4f35-4c2d-b002-4d8e3bd27c80 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:52.471833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:52.472986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:52.473273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:52.473548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] PATH_INFO: `/v3/projects/7405d44ad14342729b633b56109fe767` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:52.477803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] RBAC: Authorizing `identity:get_project(project_id=7405d44ad14342729b633b56109fe767)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:52.529527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING keystone.common.rbac_enforcer.enforcer [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 15:11:52.529870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396842e0-2a4c-4b89-aa09-605cc00cfa41 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:52.534561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Fri Jul 15 15:11:52 2022] GET /identity/v3/projects/7405d44ad14342729b633b56109fe767 => generated 344 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:11:54.525378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d236a7c-c958-489b-8c4c-b9bde629617f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:54.526249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d236a7c-c958-489b-8c4c-b9bde629617f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:54.526249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d236a7c-c958-489b-8c4c-b9bde629617f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:54.533673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:11:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:11:54.542806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:54.543292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:54.543391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:54.568943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:11:54.570488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:11:54.580000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:11:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 15:11:54.603006 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d236a7c-c958-489b-8c4c-b9bde629617f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:54.620255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d236a7c-c958-489b-8c4c-b9bde629617f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:54.625191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6758-7243-4339-b468-d65c8ffe8361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:54.626167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6758-7243-4339-b468-d65c8ffe8361 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:54.626509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6758-7243-4339-b468-d65c8ffe8361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:54.626848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6758-7243-4339-b468-d65c8ffe8361 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:54.635888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6758-7243-4339-b468-d65c8ffe8361 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:54.637705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e6758-7243-4339-b468-d65c8ffe8361 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:54.647975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Fri Jul 15 15:11:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:11:54.656163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:54.665885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f0368-9ff0-499c-a05c-9901ac1c855e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:54.671739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66acbb-029f-458b-9ac3-f69f960562c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:54.672613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66acbb-029f-458b-9ac3-f69f960562c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:54.672896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66acbb-029f-458b-9ac3-f69f960562c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:54.673168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66acbb-029f-458b-9ac3-f69f960562c4 admin admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:54.674765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66acbb-029f-458b-9ac3-f69f960562c4 admin admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:54.676672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66acbb-029f-458b-9ac3-f69f960562c4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:54.678276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Fri Jul 15 15:11:54 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:11:56.741915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c7045-567a-4e67-988c-c20b5271e41c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:56.742496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c7045-567a-4e67-988c-c20b5271e41c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:56.742496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9c7045-567a-4e67-988c-c20b5271e41c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:56.743858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:11:56.749235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:56.749669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:56.749763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:56.767388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:11:56.768578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:11:56.789999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbcf7e-1493-4327-bf86-a0a37e866410 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:56.790288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbcf7e-1493-4327-bf86-a0a37e866410 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:56.791267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dbcf7e-1493-4327-bf86-a0a37e866410 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:56.792632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:11:56.798678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:56.799136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:56.799421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:56.841004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG passlib.handlers.bcrypt [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80270) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 15:11:56.841420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG passlib.handlers.bcrypt [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80270) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 15:11:56.851854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:11:56.874821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:56.887301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:11:56.888909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:11:56.899183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:56.985946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764d902-b02f-49e4-8439-0d7c3a63d8b5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:56.993329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:11:57.015806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.024156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.024971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.025255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.025533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.037191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:11:57.048433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:11:57.049360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:11:57.050173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:11:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:11:57.055677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.118643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8e0f8-1500-4c32-a1af-b6e41db6fe79 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.120959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:57.140095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:57.160443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.161368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.161838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.161951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.168505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e613b5-14cd-4f46-a4b7-c03b442225fe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.203376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.204047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.204355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.204630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.205385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:11:57.205688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.207136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.219101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:11:57.224861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.232183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.233717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.324547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6004db-737b-4913-8410-8a6e1e7c5846 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.332425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 15:11:57.338155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.340945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:57.359086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.359974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.360256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.360528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.361760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.363294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.364985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:11:57.370367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.384775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:57.439137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce8995-0745-4a3b-9f26-4d5381cc720f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.471053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f36b6-1513-444c-b364-d76161a0ba1b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.475335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.476169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.476513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.476727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.497750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.499314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.511760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab7152-702f-4689-8620-91908110fd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.512798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab7152-702f-4689-8620-91908110fd56 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.513087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab7152-702f-4689-8620-91908110fd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.513360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab7152-702f-4689-8620-91908110fd56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.514582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab7152-702f-4689-8620-91908110fd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.516037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab7152-702f-4689-8620-91908110fd56 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.517651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:11:57.521370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:11:57.522331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:11:57.523210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:11:57.523210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:11:57.523210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:11:57.523210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:11:57.523210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:11:57.523790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab7152-702f-4689-8620-91908110fd56 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.529174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:11:57.592485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:57.606897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:11:57.627779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab7152-702f-4689-8620-91908110fd56 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.641447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209719fc-9bed-4b31-a127-207f83915863 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:11:57.664159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.664817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.665096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.665365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.666227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.domain.id=default, user.id=1b155a91a9264755b87c158c22553253) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:11:57.666544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.668229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4533889-551f-4ee3-9062-20f1875351d9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.675229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/role_assignments?user.id=1b155a91a9264755b87c158c22553253&scope.domain.id=default&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:11:57.678341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:11:57.679074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:11:57.679362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:11:57.679660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:11:57.680402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:11:57.680712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:11:57.682321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcaef6d-3fa0-4f9e-a576-668e5f42d809 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:11:57.703971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:00.135052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d8765-bcff-4197-86e5-be093ed98f7f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.135592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d8765-bcff-4197-86e5-be093ed98f7f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.135592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d8765-bcff-4197-86e5-be093ed98f7f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.137423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:00.152608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17e4ab-b3c9-4e3b-83b4-1af4e65fdb2f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.153537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17e4ab-b3c9-4e3b-83b4-1af4e65fdb2f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.154022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17e4ab-b3c9-4e3b-83b4-1af4e65fdb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.175957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ec17e4ab-b3c9-4e3b-83b4-1af4e65fdb2f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:00.177490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ec17e4ab-b3c9-4e3b-83b4-1af4e65fdb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:00.224776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb746e-4103-4bf5-9f30-e1f760e917d3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.225396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb746e-4103-4bf5-9f30-e1f760e917d3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.225879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb746e-4103-4bf5-9f30-e1f760e917d3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.227503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 13/23] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:00.241764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.242392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.242889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.281423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:00.283039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:00.311511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 11/24] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:00.313228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27812da-3f33-42c0-946f-4a214d6c7708 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.314554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27812da-3f33-42c0-946f-4a214d6c7708 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.316558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27812da-3f33-42c0-946f-4a214d6c7708 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.318729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 12/25] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:00.327973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.328547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.329039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.361346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:00.362596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:00.411761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 14/26] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 15 15:12:00.412506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:00.446874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:00.469923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 13/27] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:00.475515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:00.558721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:00.592917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ecdf2-2f2a-4416-a32f-8640ab86d4da None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:00.653339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:00.654573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.655100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.655587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.667580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907c64e-9c8e-4550-8136-7784c8997f2e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:00.669646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:00.671653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:12:00.723007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:00.724331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:00.725633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:12:00.725633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:12:00.725633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:12:00.725633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:00.725633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 15/28] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:00.725633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:00.749859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:00.751517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.752011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.752514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.768343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:00.780187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:00.781503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:00.782689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:00.785298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 14/29] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:00.786697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:00.832310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:00.908060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ea24f-9afa-4716-bf14-92503cd2601c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:00.951138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:00.952480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:00.952920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:00.953326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:00.954560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:00.955166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:00.957326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:00.958705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ee802-46b4-48fc-afce-b80a8469b6d8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:00.972065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 15/30] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:00.975502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:00.994615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:01.006345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62895c27-3708-4602-be8c-7ce15b00bd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.007570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62895c27-3708-4602-be8c-7ce15b00bd8d None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.007859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62895c27-3708-4602-be8c-7ce15b00bd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.008137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62895c27-3708-4602-be8c-7ce15b00bd8d None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.009362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62895c27-3708-4602-be8c-7ce15b00bd8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.011120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62895c27-3708-4602-be8c-7ce15b00bd8d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.087169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:01.095235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005fae0-23cf-438a-af91-931f39c2105c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:01.132607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.133990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.134725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.135243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.136346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:01.137197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.138958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.153443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 16/31] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:01.154055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 16/32] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Jul 15 15:12:00 2022] POST /identity/v3/roles => generated 219 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:01.155427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:01.160861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edce06-7496-4075-ab47-f47966fc5a72 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.161230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edce06-7496-4075-ab47-f47966fc5a72 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.161517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edce06-7496-4075-ab47-f47966fc5a72 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.167114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 17/33] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:01 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:01.169505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edce06-7496-4075-ab47-f47966fc5a72 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:01.353602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7399dc-19f4-41c4-8c6b-f73310e228e8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:01.378603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edce06-7496-4075-ab47-f47966fc5a72 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:01.403126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305daa98-bcda-4522-ad78-b6410e89bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.403802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305daa98-bcda-4522-ad78-b6410e89bbf7 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.404090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305daa98-bcda-4522-ad78-b6410e89bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.404371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305daa98-bcda-4522-ad78-b6410e89bbf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.405500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305daa98-bcda-4522-ad78-b6410e89bbf7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.406828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305daa98-bcda-4522-ad78-b6410e89bbf7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.416852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646528ec-c203-4a50-a7e0-20b828630b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.417964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646528ec-c203-4a50-a7e0-20b828630b88 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.418409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646528ec-c203-4a50-a7e0-20b828630b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.418846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646528ec-c203-4a50-a7e0-20b828630b88 None admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.420350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646528ec-c203-4a50-a7e0-20b828630b88 None admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.422074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646528ec-c203-4a50-a7e0-20b828630b88 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.423908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 18/34] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:01 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:01.425377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9b930-c6dc-47e9-bb76-2c00cf73b4e3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.425846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9b930-c6dc-47e9-bb76-2c00cf73b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.426123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9b930-c6dc-47e9-bb76-2c00cf73b4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.449483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0de9b930-c6dc-47e9-bb76-2c00cf73b4e3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:01.451281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0de9b930-c6dc-47e9-bb76-2c00cf73b4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:01.502842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:01.530542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 19/35] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:01.530542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cabe21-13f8-465b-9e37-bfc60acfd7f4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.530542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cabe21-13f8-465b-9e37-bfc60acfd7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.530542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cabe21-13f8-465b-9e37-bfc60acfd7f4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.530542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 20/36] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:01.530542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cabe21-13f8-465b-9e37-bfc60acfd7f4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:01.550790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 17/37] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:12:01 2022] POST /identity/v3/projects => generated 301 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:01.552593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.552900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.553183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.572002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:01.573670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:01.577989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:01.661572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 18/38] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:01.662540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:01.702434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cabe21-13f8-465b-9e37-bfc60acfd7f4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:01.741929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.742802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.743085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.743366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.744577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.746448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.748308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 21/39] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:01.748901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:01.787169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df86a91-d377-4b08-bd0d-befc143cbaef None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:01.807228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:01.821242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af73fa3-9b33-4222-9f09-ab4f93184afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.821959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af73fa3-9b33-4222-9f09-ab4f93184afd None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.822319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af73fa3-9b33-4222-9f09-ab4f93184afd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.822635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af73fa3-9b33-4222-9f09-ab4f93184afd None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.823376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af73fa3-9b33-4222-9f09-ab4f93184afd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.825136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af73fa3-9b33-4222-9f09-ab4f93184afd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.834486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:01.841012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 19/40] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Jul 15 15:12:01 2022] POST /identity/v3/roles => generated 225 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:01.842671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bab78-9704-48c7-98c8-de66d55454aa None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.843057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bab78-9704-48c7-98c8-de66d55454aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.843399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2bab78-9704-48c7-98c8-de66d55454aa None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.845334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 20/41] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:01.846124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bab78-9704-48c7-98c8-de66d55454aa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:01.856045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a613d-d28d-45a6-8c77-0588074d7c42 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:01.903316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974dbf-2fb8-47b6-bf3e-72d910a54dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:01.904028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974dbf-2fb8-47b6-bf3e-72d910a54dd4 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.904313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974dbf-2fb8-47b6-bf3e-72d910a54dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.904588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974dbf-2fb8-47b6-bf3e-72d910a54dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.905787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed974dbf-2fb8-47b6-bf3e-72d910a54dd4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:01.907576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed974dbf-2fb8-47b6-bf3e-72d910a54dd4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:01.924953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 22/42] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Jul 15 15:12:01 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:01.926623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:01.927109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:01.927565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:01.954456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:01.955988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:01.990068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2bab78-9704-48c7-98c8-de66d55454aa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:02.041027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:02.042642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:02.042964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:02.043345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] PATH_INFO: `/v3/domains/default/users/1b155a91a9264755b87c158c22553253/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:02.047526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1b155a91a9264755b87c158c22553253, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:02.048667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:02.057096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 23/43] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:02.067557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 21/44] 10.222.0.40 () {58 vars in 1361 bytes} [Fri Jul 15 15:12:01 2022] PUT /identity/v3/domains/default/users/1b155a91a9264755b87c158c22553253/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:02.080304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:02.307676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:02.368968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c557d-7945-4acd-afb3-7dd190bc5534 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:02.404881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f360f5-cdda-437d-9142-a6a11b9bf0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:02.405682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f360f5-cdda-437d-9142-a6a11b9bf0ff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:02.405966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f360f5-cdda-437d-9142-a6a11b9bf0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:02.406243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f360f5-cdda-437d-9142-a6a11b9bf0ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:02.407464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f360f5-cdda-437d-9142-a6a11b9bf0ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:02.408948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f360f5-cdda-437d-9142-a6a11b9bf0ff None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:02.410600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 24/45] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:02.416602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:02.589372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:02.634702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26c5a9-3c90-4c8b-a060-4390a2baa392 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:02.675195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ef452-cbfe-42b0-bc4b-ce79c58fab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:02.675869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef452-cbfe-42b0-bc4b-ce79c58fab5b None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:02.676156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef452-cbfe-42b0-bc4b-ce79c58fab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:02.676435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ef452-cbfe-42b0-bc4b-ce79c58fab5b None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:02.677189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef452-cbfe-42b0-bc4b-ce79c58fab5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:02.678675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ef452-cbfe-42b0-bc4b-ce79c58fab5b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:02.705020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 22/46] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:12:02 2022] POST /identity/v3/projects => generated 312 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:04.133040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cd16e-dd79-4862-8ad6-778fc1580b65 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.133580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cd16e-dd79-4862-8ad6-778fc1580b65 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.133661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cd16e-dd79-4862-8ad6-778fc1580b65 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.135117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 25/47] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:04.140630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.141082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.141164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.158575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:04.159792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:04.229689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 23/48] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:04.252917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cd16e-dd79-4862-8ad6-778fc1580b65 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.361231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cd16e-dd79-4862-8ad6-778fc1580b65 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:04.372183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:04.396686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:04.397486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.397768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.398040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.408266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:04.415818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:04.416688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:04.417533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 26/49] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:04.423404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.470116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:04.526315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253e681-41a8-4df3-8bbf-b661b6ad6510 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:04.563343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:04.564026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.564314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.564591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.565356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:04.565666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:04.567111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb04c1-2379-44e5-acf9-e61968dbb033 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:04.575485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 24/50] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:04.581424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.684187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe0daf-4fe5-4e1d-99b1-c141175f9974 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:04.695898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e6d50-b2f0-436b-a177-54b71d6c1900 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.696197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e6d50-b2f0-436b-a177-54b71d6c1900 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.696482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e6d50-b2f0-436b-a177-54b71d6c1900 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.697902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 25/51] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:04.703004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.703298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.703587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.717568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:04.718469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.718779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.719051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.720239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:04.720972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:04.721699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:04.722200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:04.723381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 27/52] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:04 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:04.728934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.771792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:04.793885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 26/53] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:04.816684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.837392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1c87e-586b-4ca5-8bd9-b1dbf0afd157 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:04.850428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:04.872880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:04.873935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.874219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.874876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.878157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:04.878157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:04.879375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 28/54] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:04.885296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.925855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34977b2-2297-483d-b969-fc165c4c3f48 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:04.959922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:04.960732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:04.961015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:04.961292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:04.962485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:04.964059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:04.965775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 27/55] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:04.971350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:04.987753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0d3b4-0a42-4cd0-a2ba-464bf47f90b7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:04.995190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:05.023809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:05.024459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:05.024765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:05.025050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:05.025890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.domain.id=default, user.id=1b155a91a9264755b87c158c22553253) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:05.026199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:05.028097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aac71e-3bd8-4f52-aa24-036444bcaf80 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:05.034031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 29/56] 10.222.0.40 () {56 vars in 1322 bytes} [Fri Jul 15 15:12:04 2022] GET /identity/v3/role_assignments?user.id=1b155a91a9264755b87c158c22553253&scope.domain.id=default&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:05.075686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bba717-1d3f-44ab-958a-7aa689d68847 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:05.077326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:05.114268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a7706-fa26-4c01-9580-17246e182623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:05.115004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7706-fa26-4c01-9580-17246e182623 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:05.115295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7706-fa26-4c01-9580-17246e182623 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:05.115571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a7706-fa26-4c01-9580-17246e182623 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:05.116310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a7706-fa26-4c01-9580-17246e182623 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:05.117804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a7706-fa26-4c01-9580-17246e182623 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:05.147655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 28/57] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Jul 15 15:12:04 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:12:07.164351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64f33-116d-4aa4-b65e-7ccf6e903fe4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:07.165192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64f33-116d-4aa4-b65e-7ccf6e903fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:07.165192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba64f33-116d-4aa4-b65e-7ccf6e903fe4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:07.167623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 30/58] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:07.173954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:07.174714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:07.174800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:07.200737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:07.201969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:07.280548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 29/59] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:07.305736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64f33-116d-4aa4-b65e-7ccf6e903fe4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:07.433682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba64f33-116d-4aa4-b65e-7ccf6e903fe4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:07.471108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752acc1-c94b-4f1a-9762-ba14c5f0be02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:07.471863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752acc1-c94b-4f1a-9762-ba14c5f0be02 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:07.472143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752acc1-c94b-4f1a-9762-ba14c5f0be02 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:07.472416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752acc1-c94b-4f1a-9762-ba14c5f0be02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:07.473808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752acc1-c94b-4f1a-9762-ba14c5f0be02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:07.475347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752acc1-c94b-4f1a-9762-ba14c5f0be02 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:07.477013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 31/60] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:07.483215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:07.575782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:07.611116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877f213-5e91-4f45-9ad0-f0cf885ca079 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:07.661456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e79b1f-951f-47cb-8de0-97a924e45e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:07.662203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e79b1f-951f-47cb-8de0-97a924e45e8e None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:07.662491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e79b1f-951f-47cb-8de0-97a924e45e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:07.662818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e79b1f-951f-47cb-8de0-97a924e45e8e None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:07.663567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e79b1f-951f-47cb-8de0-97a924e45e8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:07.665047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e79b1f-951f-47cb-8de0-97a924e45e8e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:07.692347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 30/61] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:07 2022] POST /identity/v3/users => generated 275 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:10.813262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd010ef-e228-4c3f-86c4-f10475c764b9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:10.814368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd010ef-e228-4c3f-86c4-f10475c764b9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:10.814904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd010ef-e228-4c3f-86c4-f10475c764b9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:10.816526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 32/62] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:10.822338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbd208-676e-4269-93ec-df5b54b90d14 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:10.822786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbd208-676e-4269-93ec-df5b54b90d14 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:10.822932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbd208-676e-4269-93ec-df5b54b90d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:10.845293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-23cbd208-676e-4269-93ec-df5b54b90d14 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:10.847026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23cbd208-676e-4269-93ec-df5b54b90d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:10.944311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 31/63] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:10.992145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd010ef-e228-4c3f-86c4-f10475c764b9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.003113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b0005-79fc-469d-bc93-e63c06d56fe2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.003447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b0005-79fc-469d-bc93-e63c06d56fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.003699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b0005-79fc-469d-bc93-e63c06d56fe2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.006099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 32/64] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:11.014325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.014743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.014934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.047912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:11.049217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:11.062960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:11.168853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd010ef-e228-4c3f-86c4-f10475c764b9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.195291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:11.203303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 33/65] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:11.229653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.230391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.230961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.231433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.231868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.264097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:11.273388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:11.274243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:11.275146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:11.277844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 33/66] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:11.287491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.358493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa25f7-abda-4d9c-a3d6-d69da268dde7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.404466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7a7ce-095f-49f3-9506-a87d2dd22dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.405627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7a7ce-095f-49f3-9506-a87d2dd22dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.406094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7a7ce-095f-49f3-9506-a87d2dd22dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.406391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7a7ce-095f-49f3-9506-a87d2dd22dc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.419051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7a7ce-095f-49f3-9506-a87d2dd22dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:11.425936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f6e7a7ce-095f-49f3-9506-a87d2dd22dc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:11.426770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:11.427676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:11.428537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 34/67] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:11.429735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201703d-fa26-4d98-944c-2c6d1a1267a1 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.429827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201703d-fa26-4d98-944c-2c6d1a1267a1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.430087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201703d-fa26-4d98-944c-2c6d1a1267a1 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.431533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 35/68] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:11.432674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8ab73-bfc1-4830-b823-b21773a53a3a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.433003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8ab73-bfc1-4830-b823-b21773a53a3a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.433293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8ab73-bfc1-4830-b823-b21773a53a3a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.434114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48853-682c-40b0-bc24-373f8a5c4cbe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.438565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 36/69] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:11.438565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8ab73-bfc1-4830-b823-b21773a53a3a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.482649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.483379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.483668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.483954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.485242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:11.485567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:11.487060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ac8355-cee3-4a31-9e0d-76a29efa356e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:11.497210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 34/70] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:11.498954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82a3e2-e75b-45ae-8c8d-24cbb6ee9950 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.499714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82a3e2-e75b-45ae-8c8d-24cbb6ee9950 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.500002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82a3e2-e75b-45ae-8c8d-24cbb6ee9950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.509087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:11.520215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9c82a3e2-e75b-45ae-8c8d-24cbb6ee9950 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:11.521480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9c82a3e2-e75b-45ae-8c8d-24cbb6ee9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:11.550771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8ab73-bfc1-4830-b823-b21773a53a3a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.584679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.585357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.585640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.585919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.586756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:11.587058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:11.588288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 35/71] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:11.588473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:11.589655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec609565-8225-48d3-81fb-91c443ec7182 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.589973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec609565-8225-48d3-81fb-91c443ec7182 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.590245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec609565-8225-48d3-81fb-91c443ec7182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.606396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ec609565-8225-48d3-81fb-91c443ec7182 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:11.607606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ec609565-8225-48d3-81fb-91c443ec7182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:11.613757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 37/72] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:11.613757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.677707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 36/73] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:11.679163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60befa3b-d672-433f-b7ea-04d99477e481 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.679463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60befa3b-d672-433f-b7ea-04d99477e481 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.679745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60befa3b-d672-433f-b7ea-04d99477e481 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.681069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 37/74] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:11.682218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60befa3b-d672-433f-b7ea-04d99477e481 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.704832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:11.728998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b2bcea-475c-4a1a-aa4f-afaded292638 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.760058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:11.773458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.774369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.774703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.774979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.792215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:11.792818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60befa3b-d672-433f-b7ea-04d99477e481 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.794677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:11.796639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 38/75] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:11.797451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.828179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09c9a3-cd4a-47e2-aca8-b74bead25684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.829013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09c9a3-cd4a-47e2-aca8-b74bead25684 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.829285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09c9a3-cd4a-47e2-aca8-b74bead25684 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.829555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09c9a3-cd4a-47e2-aca8-b74bead25684 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.840346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09c9a3-cd4a-47e2-aca8-b74bead25684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-db09c9a3-cd4a-47e2-aca8-b74bead25684 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:11.848309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:11.849165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:11.849942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 38/76] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:11.850819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.850989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.851262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.868611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:11.869809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:11.908399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ec9bf-ac8c-47ac-850a-973c1f446931 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:11.938242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 39/77] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:11.939265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.945791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:11.946814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:11.947174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:11.947529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:11.948892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:11.950925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:11.952640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 39/78] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:11.953460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:11.963281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.007308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.048588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613c211-d13c-46ad-b1eb-a65c5ff3199d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.059984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a083a5-0cad-41a7-ad81-77781569e004 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.082765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.083649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.083932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.084211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.094197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.094886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.094988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:12.095183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.095462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:12.102635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:12.103573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 40/79] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:12.109538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.168628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.170267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.180034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.212142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c080c-c751-4633-8205-426c85b81431 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.218176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.246686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.247361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.247632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.247911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.248984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:12.249284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.250774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.264078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 41/80] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:12.264078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.271039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 40/81] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:12.271924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.361710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96a905-b73a-4999-8880-e48b50c4b2a9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.367200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f990ec17-5607-49ad-a901-48b9fbdfaadb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.390738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.396524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.397192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.397485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.397752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.400645 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.401458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.401775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.402064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.403391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.404855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.406508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 41/82] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:12.407323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.425246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.467769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.469318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.510770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df5bbf-2528-4cfd-9db3-9159cf94c19b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.546056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.546795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.547084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.547336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.548100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:12.548414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.550034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.558866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 42/83] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:12.559427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.573680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 42/84] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:12.574644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.610115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.642182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.663856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce7902-c34a-4bf5-a939-a4d8a9eb4226 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.680883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ee78f-f4b7-40d0-b85e-894d6c0b6169 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.698325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.699764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.700118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.700395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.701647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.703317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.705189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 43/85] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:12.705778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.715498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.716382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.716662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.716935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.718195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.719643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.721373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 43/86] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:12.722215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.810405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64e842-b34e-407b-a4ad-04c0132899e9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.819378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.822966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78435f7-6268-46c6-b066-562e8cdbb59a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.845055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.845737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.846027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.846316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.847064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.848535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.857506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:12.858381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:12.858691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:12.858969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:12.860410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:12.861963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:12.863702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 44/87] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:12.882547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 44/88] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:12 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:12.882798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.882880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:12.882880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:12.991032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e625c76-c616-47b4-9b8a-528d3a074404 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:12.999517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2edd7a-16b5-49be-af39-93fc0f7d37c5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.028537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.029524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.029807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.030085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.032324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.032785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.034760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 45/89] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:13.035287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:13.052590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.053259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.053562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.053839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.073974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:13.141292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.143314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.148170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:13.156211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562af0f-91de-466c-bf8b-7a31ac7e5b4d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.203647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.204315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.204603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.204882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.206274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=d198e1a875754bcc9e77e87149e160fd) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:13.206662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.208659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.217611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 46/90] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/role_assignments?user.id=d198e1a875754bcc9e77e87149e160fd&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:13.218627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:13.262227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 45/91] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:13.263047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:13.339224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:13.342375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ddc75-0937-4237-90e9-678d91921baa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.378815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e3eff-0deb-4a42-801e-134251b16fb7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.380315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:13.381713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.382378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.382722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.382996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.436385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.437261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.437551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.437831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.438772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=1b155a91a9264755b87c158c22553253) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:13.439190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.440688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.453005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 46/92] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/role_assignments?user.id=1b155a91a9264755b87c158c22553253&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:13.453866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:13.462106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.464095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.604031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5abac-18e4-4779-b1db-1bcfac8d490f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.642418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.643351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.643643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.643926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.645208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.650544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.650544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 47/93] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:13.655514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:13.688951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:13.693595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 47/94] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:13.701860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:13.718917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:13.817844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65829a-a0d7-490f-86ba-c665fcd8349e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.838407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad70b3-f8fd-4df6-bea5-34799b48adc2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:13.855011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.855599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.855886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.856201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.857013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] RBAC: Adding query filter params (role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=d198e1a875754bcc9e77e87149e160fd) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:13.857327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.858829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.865048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 48/95] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/role_assignments?user.id=d198e1a875754bcc9e77e87149e160fd&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:13.880845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc30c1f-a556-4322-9887-06b4496a0b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:13.882375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc30c1f-a556-4322-9887-06b4496a0b6c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:13.882836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc30c1f-a556-4322-9887-06b4496a0b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:13.883398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc30c1f-a556-4322-9887-06b4496a0b6c None admin] PATH_INFO: `/v3/projects/7c571694986b441d9d725f8df1330686` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:13.884746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc30c1f-a556-4322-9887-06b4496a0b6c None admin] RBAC: Authorizing `identity:get_project(project_id=7c571694986b441d9d725f8df1330686)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:13.886782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc30c1f-a556-4322-9887-06b4496a0b6c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:13.888750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/projects/7c571694986b441d9d725f8df1330686 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:13.895064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:14.003338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:14.007277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082daa9-339a-4040-8aa3-8656ac6c45af None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:14.043864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:14.044552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:14.044839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:14.045121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:14.045969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=7c571694986b441d9d725f8df1330686, user.id=d198e1a875754bcc9e77e87149e160fd) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:14.046285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:14.047794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f437e-c8ec-4f83-ab98-d2e391a1c2ef None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:14.053933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:13 2022] GET /identity/v3/role_assignments?user.id=d198e1a875754bcc9e77e87149e160fd&scope.project.id=7c571694986b441d9d725f8df1330686&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:15.528661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4a578-cdf0-4d7d-b028-89f836933753 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:15.529707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4a578-cdf0-4d7d-b028-89f836933753 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:15.530203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4a578-cdf0-4d7d-b028-89f836933753 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:15.531826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:15.537390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:15.537796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:15.537950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:15.559894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:15.561462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:15.648378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 50/99] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:15.688373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4a578-cdf0-4d7d-b028-89f836933753 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:15.791420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:15.838440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4a578-cdf0-4d7d-b028-89f836933753 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:15.882762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26c115-5a19-42d2-a26f-ea84a83b9b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:15.883597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26c115-5a19-42d2-a26f-ea84a83b9b13 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:15.883883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26c115-5a19-42d2-a26f-ea84a83b9b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:15.884152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26c115-5a19-42d2-a26f-ea84a83b9b13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:15.900045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26c115-5a19-42d2-a26f-ea84a83b9b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7a26c115-5a19-42d2-a26f-ea84a83b9b13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:15.907849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:15.908693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:15.909431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 50/100] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:15.919717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.069782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:16.079043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce96f2-b827-4ea0-a3a4-c5e346680730 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:16.089603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4fcac-027e-43b3-bc11-555dbd12267f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.089904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4fcac-027e-43b3-bc11-555dbd12267f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.090159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4fcac-027e-43b3-bc11-555dbd12267f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.091669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 51/101] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:16.100801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859537a-38e5-4271-b3fe-4bcdd0f2aeca None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.101400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859537a-38e5-4271-b3fe-4bcdd0f2aeca None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.101873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859537a-38e5-4271-b3fe-4bcdd0f2aeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.126547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:16.126547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.126547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.126547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.126910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:16.126910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:16.126910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:16.129899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1859537a-38e5-4271-b3fe-4bcdd0f2aeca None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:16.133589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1859537a-38e5-4271-b3fe-4bcdd0f2aeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:16.140453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 51/102] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:16.149856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.243917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 52/103] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:16.285542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb4ab-91b2-4bc8-acdf-d6461359b996 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.286192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb4ab-91b2-4bc8-acdf-d6461359b996 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.286704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435bb4ab-91b2-4bc8-acdf-d6461359b996 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.288320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 53/104] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:16.293857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.294384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.294910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.313896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:16.315491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:16.357217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a207c973-caf0-4af1-843d-ad40ceff90b8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:16.398944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:16.420810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 54/105] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:16.422879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:16.423787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.424073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.424355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] PATH_INFO: `/v3/users/a9ed86ea99fc4faaa721ac5da51c1cbc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.425236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.455898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] RBAC: Authorizing `identity:get_user(user_id=a9ed86ea99fc4faaa721ac5da51c1cbc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:16.457787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:16.460126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 52/106] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:16 2022] GET /identity/v3/users/a9ed86ea99fc4faaa721ac5da51c1cbc => generated 289 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:16.461098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.482668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:16.599847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579679c-396f-4fbc-a588-2991e641cb93 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:16.607322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d619ac-018d-4d3d-8b20-0c4ca72a0ba5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:16.645993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69359e8f-5a7d-4c59-896a-3cca65dd8c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:16.647266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359e8f-5a7d-4c59-896a-3cca65dd8c4b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.647775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359e8f-5a7d-4c59-896a-3cca65dd8c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.648279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69359e8f-5a7d-4c59-896a-3cca65dd8c4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.653571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:16.654490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.654717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.655005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.661025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69359e8f-5a7d-4c59-896a-3cca65dd8c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:16.666286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-69359e8f-5a7d-4c59-896a-3cca65dd8c4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:16.669945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:16.670837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:16.671616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:16.679061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:16.680005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 55/108] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:16.680774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.681456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 53/108] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:16.681456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c587ce-4b66-4533-8691-54778f79442c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.681456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c587ce-4b66-4533-8691-54778f79442c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.681456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c587ce-4b66-4533-8691-54778f79442c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.684424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 54/109] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:16.684424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c587ce-4b66-4533-8691-54778f79442c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.736151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:16.772668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:16.793711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843c235-ea0a-4b42-9a49-c6b4313a1b5d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:16.808059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c587ce-4b66-4533-8691-54778f79442c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:16.831282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc582800-121f-4c56-b087-7a7733ac3429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:16.832373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc582800-121f-4c56-b087-7a7733ac3429 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.832869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc582800-121f-4c56-b087-7a7733ac3429 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.833340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc582800-121f-4c56-b087-7a7733ac3429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.847804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:16.848517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:16.848807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:16.849090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:16.849951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:16.850268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:16.851851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:16.861872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 55/110] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:16.862476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:16.908465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc582800-121f-4c56-b087-7a7733ac3429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:16.909983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc582800-121f-4c56-b087-7a7733ac3429 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:16.966769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:16.976422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab89a5c-5ddd-487e-98f4-897b2c2882eb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.012596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.013273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.013558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.014002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.014612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:17.014945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.016413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.017898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.020579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 56/111] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:17.025651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494241f9-d9fc-4f78-b89e-f6e72add18a3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.025651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494241f9-d9fc-4f78-b89e-f6e72add18a3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.025651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494241f9-d9fc-4f78-b89e-f6e72add18a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.034701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 56/112] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.035584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.043875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-494241f9-d9fc-4f78-b89e-f6e72add18a3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:17.045098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-494241f9-d9fc-4f78-b89e-f6e72add18a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:17.120394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 57/113] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:17.122402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2316d-20d4-44c5-a641-4eca0f706f72 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.122726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2316d-20d4-44c5-a641-4eca0f706f72 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.123530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2316d-20d4-44c5-a641-4eca0f706f72 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.125501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 58/114] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:17.129694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2316d-20d4-44c5-a641-4eca0f706f72 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.143022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eadceb-6c16-453e-9f2b-8fac321cc617 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.177040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.177934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.178214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.178489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.179705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.181157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.182963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 57/115] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.187640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.210387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.233507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2316d-20d4-44c5-a641-4eca0f706f72 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.249328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.268408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb983612-0c69-414c-86a9-f0e0b0207177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.269284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb983612-0c69-414c-86a9-f0e0b0207177 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.269565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb983612-0c69-414c-86a9-f0e0b0207177 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.269845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb983612-0c69-414c-86a9-f0e0b0207177 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.271065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb983612-0c69-414c-86a9-f0e0b0207177 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.272554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb983612-0c69-414c-86a9-f0e0b0207177 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.274267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 59/116] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.275466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298830c-706e-4770-9de0-f1c7ba4b6fd1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.275748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298830c-706e-4770-9de0-f1c7ba4b6fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.276182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298830c-706e-4770-9de0-f1c7ba4b6fd1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.277999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 60/117] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:17.279078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.279365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.279645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.292766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b423f28-1831-4b36-bf1a-df650ee15061 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.297004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:17.298218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:17.327575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.328470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.328761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.329037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.330272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.331789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.333540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 58/118] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.334693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.366313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 61/119] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:17.367269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.442192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.446509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e659b-7ce5-4ac5-ba5f-295cfe503116 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.466539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba6782-d327-49fa-ae35-a2d4455d773c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.475519 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.480986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.481815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.482100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.482377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.483567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.485025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.486846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 59/120] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.487427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.504968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c512bc1-5faf-4e5c-a2e7-b72861fbaa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.505836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c512bc1-5faf-4e5c-a2e7-b72861fbaa63 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.506116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c512bc1-5faf-4e5c-a2e7-b72861fbaa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.506391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c512bc1-5faf-4e5c-a2e7-b72861fbaa63 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.507595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c512bc1-5faf-4e5c-a2e7-b72861fbaa63 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.509095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c512bc1-5faf-4e5c-a2e7-b72861fbaa63 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.510903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 62/121] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.512111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.512393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.512667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.529974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:17.531210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:17.590108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682598c-8df6-46d2-a767-d68aca0bbee3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.599419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 63/122] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:17.600443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.623707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.624369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.624655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.624930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.625763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] RBAC: Adding query filter params (role.id=dd25282bc3de4eb296ccdc93ac09b781, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=a9ed86ea99fc4faaa721ac5da51c1cbc) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:17.626077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.627563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.633071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 60/123] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/role_assignments?user.id=a9ed86ea99fc4faaa721ac5da51c1cbc&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=dd25282bc3de4eb296ccdc93ac09b781 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.633963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.702156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.726999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.732063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d128714-6ee8-467a-966c-905501a0b374 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.770878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2023e6a1-bf2c-45fb-abda-eca16c047c2d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.773106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.774074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.774348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.774680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.775954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.777625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.779421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 64/124] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:17.784879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.817931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.818794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.819085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.819364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.832259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:17.841002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:17.842010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:17.842822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 61/125] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:17.843448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:17.896946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f65e05a-de60-4d55-b079-3303c87510df None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.935019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:17.935699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:17.936082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:17.936363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:17.937580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:17.939222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:17.955901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:17.960670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17210-6b1e-4c85-8182-f2580be1b32e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:17.971966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 65/126] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:12:17 2022] POST /identity/v3/projects => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:17.973357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.001546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:18.018841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.020134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.020619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.021115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977/users/1b155a91a9264755b87c158c22553253/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.024942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7c450055241099126ea71cabc3977, user_id=1b155a91a9264755b87c158c22553253, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.027725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.049024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 62/127] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:17 2022] PUT /identity/v3/projects/98d7c450055241099126ea71cabc3977/users/1b155a91a9264755b87c158c22553253/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:18.050193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.128257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345271db-1204-4139-92d9-ad11dffcaedc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.169797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.171040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.171518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.172001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.184951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:18.194132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:18.195089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:18.195860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:18.198905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 66/128] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:18.199974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.203073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d886f56-8799-42a6-94be-94fff059cbe3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.249636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.250245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.250646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.251001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977/users/d198e1a875754bcc9e77e87149e160fd/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.253232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7c450055241099126ea71cabc3977, user_id=d198e1a875754bcc9e77e87149e160fd, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.254857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.275040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 63/129] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:18 2022] PUT /identity/v3/projects/98d7c450055241099126ea71cabc3977/users/d198e1a875754bcc9e77e87149e160fd/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:18.275929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:18.276921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.341031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:18.350422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9794d2a3-9821-4bfd-ac2f-751f91cc9dff None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.398492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.399233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.399522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.399804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.400596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:18.400974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.402467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.412758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 67/130] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:18.418604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.488825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6d59a-f18a-41ca-bfb6-1f07fdc0b9bb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.537149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39a6fc-882d-4846-94ef-b57b0b39b715 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.573293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.573988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.574271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.574579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.575348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:18.575656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.577048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.580884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:18.586101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 64/131] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:18.588492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.589608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.590026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.590312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.592359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.593197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.594273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.596387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 68/132] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:18 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:18.601909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.726557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadda89-dd91-46ee-b134-5f4013b71383 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.764834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.766031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.766407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.766951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.768620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.770367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.772541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 69/133] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:18 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:18.778934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.780343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b9ef3-efe7-4664-b16e-62ccfed3a6da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:18.793301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:18.883708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:18.885020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:18.885656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:18.886133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:18.888183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:18.889873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:18.892217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 65/134] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:18 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:18.897499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:18.898858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:19.048629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341266e-c845-4125-be08-4b33fef8dd18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:19.068392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92742468-b9fb-40b8-af3a-f9e66936ab00 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:19.103139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:19.104462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:19.104948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:19.105420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] PATH_INFO: `/v3/projects/7c571694986b441d9d725f8df1330686` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:19.111307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] RBAC: Authorizing `identity:get_project(project_id=7c571694986b441d9d725f8df1330686)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:19.113032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:19.115040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 66/135] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:18 2022] GET /identity/v3/projects/7c571694986b441d9d725f8df1330686 => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:19.121099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:19.139939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d65ee-a551-4cd7-bb9d-e4947639fd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:19.141228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d65ee-a551-4cd7-bb9d-e4947639fd9f None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:19.141699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d65ee-a551-4cd7-bb9d-e4947639fd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:19.142175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d65ee-a551-4cd7-bb9d-e4947639fd9f None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977/users/d198e1a875754bcc9e77e87149e160fd/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:19.147001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d65ee-a551-4cd7-bb9d-e4947639fd9f None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7c450055241099126ea71cabc3977, user_id=d198e1a875754bcc9e77e87149e160fd, role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:19.148705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d65ee-a551-4cd7-bb9d-e4947639fd9f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:19.176576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 70/136] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:18 2022] PUT /identity/v3/projects/98d7c450055241099126ea71cabc3977/users/d198e1a875754bcc9e77e87149e160fd/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:12:19.209602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:19.253136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341aa72-730f-4a21-b93f-6e108151e519 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:19.290738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae88575-b984-4023-be08-1a872d3f355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:19.292117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88575-b984-4023-be08-1a872d3f355c None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:19.292774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88575-b984-4023-be08-1a872d3f355c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:19.293255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88575-b984-4023-be08-1a872d3f355c None admin] PATH_INFO: `/v3/projects/7c571694986b441d9d725f8df1330686/users/d198e1a875754bcc9e77e87149e160fd/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:19.295720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae88575-b984-4023-be08-1a872d3f355c None admin] RBAC: Authorizing `identity:create_grant(project_id=7c571694986b441d9d725f8df1330686, user_id=d198e1a875754bcc9e77e87149e160fd, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:19.297546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae88575-b984-4023-be08-1a872d3f355c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:19.319215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 67/137] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:19 2022] PUT /identity/v3/projects/7c571694986b441d9d725f8df1330686/users/d198e1a875754bcc9e77e87149e160fd/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:20.184098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010aeff-d99e-4d3f-ac7f-453f28c7cf19 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:20.184633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010aeff-d99e-4d3f-ac7f-453f28c7cf19 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:20.184633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010aeff-d99e-4d3f-ac7f-453f28c7cf19 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:20.186034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 71/138] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:20.192007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:20.192532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:20.192532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:20.215842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:20.217089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:20.287337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 68/139] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:20.332097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010aeff-d99e-4d3f-ac7f-453f28c7cf19 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:20.455600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010aeff-d99e-4d3f-ac7f-453f28c7cf19 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:20.475054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:20.499627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:20.500434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:20.500715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:20.500987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:20.521432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:20.529181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:20.530083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:20.530925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 72/140] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:20.538732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:20.582649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:20.670430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67128269-2375-4c01-b355-9cb8740e4f3a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:20.704626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:20.705320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:20.705606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:20.705887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:20.706675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:20.706986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:20.708377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91dfc0-b607-4f13-b896-895d0914a1cf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:20.721359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 69/141] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:20.727042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:20.844985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c3a9-d861-4440-bee8-dd59fd9c852d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:20.845519 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c3a9-d861-4440-bee8-dd59fd9c852d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:20.845993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c3a9-d861-4440-bee8-dd59fd9c852d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:20.847616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 70/142] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:20.860796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:20.861087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:20.861362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:20.896657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa66f5-39a2-4dd2-aa01-64b3ffc2c916 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:20.910463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:20.911693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:21.003352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.004182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.004468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.004873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] PATH_INFO: `/v3/users/a9ed86ea99fc4faaa721ac5da51c1cbc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.011403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] RBAC: Authorizing `identity:get_user(user_id=a9ed86ea99fc4faaa721ac5da51c1cbc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:21.012902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:21.014830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 73/143] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:20 2022] GET /identity/v3/users/a9ed86ea99fc4faaa721ac5da51c1cbc => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:21.023453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.036593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 71/144] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:21.078847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.105303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:21.137054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:21.228347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9edca-e045-4706-9963-fbc65c6f888d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:21.274637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.274637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.274637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.274637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.275058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:21.275437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:21.278470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 74/145] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:21 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:21.284561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.331569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72234b65-c4a9-4c27-b9c2-c9dbb9f713d6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:21.415789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21840f28-b5a6-448c-844f-dc8b6ad72604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.417105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21840f28-b5a6-448c-844f-dc8b6ad72604 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.417548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21840f28-b5a6-448c-844f-dc8b6ad72604 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.417967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21840f28-b5a6-448c-844f-dc8b6ad72604 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21840f28-b5a6-448c-844f-dc8b6ad72604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:21.450590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64849b6-82c2-4d36-a888-af60cacde90b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-21840f28-b5a6-448c-844f-dc8b6ad72604 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:21.462258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:21.463382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:21.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:21.467038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 72/146] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:21.468461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c47194-4a37-4584-9553-01956d43afb9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.468923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c47194-4a37-4584-9553-01956d43afb9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.469399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c47194-4a37-4584-9553-01956d43afb9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.469521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:21.472055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 73/147] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:21.479572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c47194-4a37-4584-9553-01956d43afb9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.496819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788428b-d47a-4c03-a3b5-a2b19b3dbb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.496819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788428b-d47a-4c03-a3b5-a2b19b3dbb4e None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.496819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788428b-d47a-4c03-a3b5-a2b19b3dbb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.496819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788428b-d47a-4c03-a3b5-a2b19b3dbb4e None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977/users/a9ed86ea99fc4faaa721ac5da51c1cbc/roles/dd25282bc3de4eb296ccdc93ac09b781` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.497223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788428b-d47a-4c03-a3b5-a2b19b3dbb4e None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7c450055241099126ea71cabc3977, user_id=a9ed86ea99fc4faaa721ac5da51c1cbc, role_id=dd25282bc3de4eb296ccdc93ac09b781)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:21.499017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788428b-d47a-4c03-a3b5-a2b19b3dbb4e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:21.514450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 75/148] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:21 2022] PUT /identity/v3/projects/98d7c450055241099126ea71cabc3977/users/a9ed86ea99fc4faaa721ac5da51c1cbc/roles/dd25282bc3de4eb296ccdc93ac09b781 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:12:21.520066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793216d-0baf-415a-b751-3c7fb9caf265 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.520417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793216d-0baf-415a-b751-3c7fb9caf265 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.520768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793216d-0baf-415a-b751-3c7fb9caf265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.554535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7793216d-0baf-415a-b751-3c7fb9caf265 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:21.555714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7793216d-0baf-415a-b751-3c7fb9caf265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:21.647032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 76/149] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:21.648390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c459-2e94-4184-9f13-d4b82b3c7427 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.648676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c459-2e94-4184-9f13-d4b82b3c7427 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.648952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c459-2e94-4184-9f13-d4b82b3c7427 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.655092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 77/150] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:21.662748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.663036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.663315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.677283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:21.697584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:21.699260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:21.717291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c47194-4a37-4584-9553-01956d43afb9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:21.767481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.768299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.768668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.768967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.769890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:21.770204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:21.770687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 78/151] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:21.771333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.771640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:21.792520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 74/152] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:21.798581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.800587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:21.902299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb4d89-e546-4f52-8c17-f1b80fdc92df None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:21.931464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78784f59-08f3-4c66-9317-c703615f4f73 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:21.957385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.958593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.959069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.959591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.961018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:21.962713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:21.964639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 79/153] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:21.965610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:21.984056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:21.984955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:21.985247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:21.985510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] PATH_INFO: `/v3/users/1b155a91a9264755b87c158c22553253` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:21.987938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] RBAC: Authorizing `identity:get_user(user_id=1b155a91a9264755b87c158c22553253)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:21.989416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:22.000142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 75/154] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:21 2022] GET /identity/v3/users/1b155a91a9264755b87c158c22553253 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:22.001048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:22.044087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:22.121500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:22.139932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58e81f-7dea-4b31-8478-77a8b091101b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:22.186242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:22.187470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.187959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.188421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.200615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:22.209397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:22.210260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:22.211238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:22.214633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 80/155] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:22.215671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:22.273721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641011-2bda-40d0-beef-46f9746b2342 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:22.312229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:22.312925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.313213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.313491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.314242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:22.315705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:22.335798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c84c5-35d2-481a-9ea0-671281a27c00 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:22.339434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 76/156] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:22 2022] POST /identity/v3/users => generated 283 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:22.340120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:22.383257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69533d6-3fd0-4248-948b-14b8a87acb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:22.384026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69533d6-3fd0-4248-948b-14b8a87acb54 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.384341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69533d6-3fd0-4248-948b-14b8a87acb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.384629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69533d6-3fd0-4248-948b-14b8a87acb54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.418532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:22.436783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:22.475870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf8e3c-b256-4dc2-b9af-547452c9bc3b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:22.482765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69533d6-3fd0-4248-948b-14b8a87acb54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:22.484303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69533d6-3fd0-4248-948b-14b8a87acb54 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:22.517652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:22.518490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.518816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.519090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.519853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:22.520160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:22.521785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3fb0f-ca20-4127-b417-85293a86b57e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:22.533481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 77/157] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:22.534999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83d70e-616c-42c1-8924-0f3742ff7301 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.535332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83d70e-616c-42c1-8924-0f3742ff7301 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.535658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83d70e-616c-42c1-8924-0f3742ff7301 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.536940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 78/158] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:22.541320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11795c2-1c4f-416b-b525-8c057239a510 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.541320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11795c2-1c4f-416b-b525-8c057239a510 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.541320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11795c2-1c4f-416b-b525-8c057239a510 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.541320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 79/159] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:22.541320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11795c2-1c4f-416b-b525-8c057239a510 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:22.620819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 81/160] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:22.626416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6def636-025a-42d2-acda-c6ef0ab06210 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.626416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6def636-025a-42d2-acda-c6ef0ab06210 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.626416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6def636-025a-42d2-acda-c6ef0ab06210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.645711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b6def636-025a-42d2-acda-c6ef0ab06210 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:22.647173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b6def636-025a-42d2-acda-c6ef0ab06210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:22.682202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11795c2-1c4f-416b-b525-8c057239a510 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:22.710198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:22.724591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:22.730918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:22.732017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.732660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.732941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.734164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:22.734416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 82/161] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:22.735690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:22.735874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.736081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.736358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.737665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 80/162] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:22 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:22.738446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:22.755071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:22.756264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:22.863317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 83/163] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:22.864456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:22.873073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b39d26-fd85-434a-89eb-a5d4b3c1c45f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:22.913429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:22.914697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:22.915176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:22.915623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:22.917988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:22.919874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:22.921980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 81/164] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:22 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:22.923011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.004173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7ac0b-6a8e-4884-a8e3-604e428a0f61 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.025820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:23.049465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.049856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:23.050481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.050481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.050890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.092856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bda7c2e-6345-4b3a-a10a-65c4d6aa89dd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.137460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.138616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.138872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.139197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:23.139325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.140722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:23.152711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:23.166732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:23.167746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:23.168520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 82/165] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:23.169448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.254989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 84/166] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:23.255693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.274648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:23.302690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b16d-a53a-487a-ad72-995e510c753e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.330346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:23.338456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.339358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.339649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.340194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.351231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:23.365452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:23.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:23.367222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 83/167] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:23.368000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.416295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772be8a8-f2d1-4888-9a4d-1ffb80f10edb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.483294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.483985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.484271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.484550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.485412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=1b155a91a9264755b87c158c22553253) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:23.485725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:23.487265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:23.494199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 85/168] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/role_assignments?user.id=1b155a91a9264755b87c158c22553253&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:23.495246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.606693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:23.620059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0f46a-92c1-406e-a942-8d302254214b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.620741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7eb9d-d9ed-41f8-ad4c-53fc3f3ef532 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.660466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.661852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.662142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.662425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.663755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:23.665278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:23.668000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 86/169] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:23.668939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.749054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.749741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.750020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.750297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.751086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:23.751393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:23.752782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:23.764241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 84/170] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:23.765114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:23.867311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:23.876827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10014e6-43ec-42c8-8750-b03befe63291 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.965764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:23.966428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ace018-91cb-4851-91a1-c601805c07a5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:23.966570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:23.966737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:23.967017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:23.967790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:23.968097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:23.969565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:23.982436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 87/171] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:23.983394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:24.008021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:24.030353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:24.031551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.032109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.032615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.033700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=d198e1a875754bcc9e77e87149e160fd) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:24.034358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:24.036206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:24.048448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 85/172] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/role_assignments?user.id=d198e1a875754bcc9e77e87149e160fd&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 608 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:24.049641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:24.192767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa70ec-aa3a-4362-839f-8adfc9575add None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:24.218299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d056c-e92f-4728-9cd3-946f5a87915e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:24.238863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:24.268331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:24.270069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.271004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.271822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.273722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:24.274650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:24.275996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.278084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:24.278651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.279126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] PATH_INFO: `/v3/users/d198e1a875754bcc9e77e87149e160fd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.282023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d198e1a875754bcc9e77e87149e160fd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:24.283771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:24.284417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 86/173] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:24 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:24.286602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 88/174] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:23 2022] GET /identity/v3/users/d198e1a875754bcc9e77e87149e160fd => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:24.290045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:24.299650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:24.427147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c0038-22fb-4297-8e1c-8f9a81b56358 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:24.467328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee52b5-06d7-4c48-84bb-6356693019a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:24.468417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee52b5-06d7-4c48-84bb-6356693019a6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.468899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee52b5-06d7-4c48-84bb-6356693019a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.469389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee52b5-06d7-4c48-84bb-6356693019a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.474222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:24.522274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a36750-1f7b-4a6f-95f7-9dceace648f5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:24.544607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:24.568916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee52b5-06d7-4c48-84bb-6356693019a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:24.570471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee52b5-06d7-4c48-84bb-6356693019a6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:24.615839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:24.616796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.617222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.617663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.699868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 87/175] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:24.706913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f90b-d81a-4bf3-bd2c-12dd46f17e2e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.707201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f90b-d81a-4bf3-bd2c-12dd46f17e2e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.707482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f90b-d81a-4bf3-bd2c-12dd46f17e2e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.708986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 88/176] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:24.709550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f90b-d81a-4bf3-bd2c-12dd46f17e2e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:24.804957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:24.807055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:24.820008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f90b-d81a-4bf3-bd2c-12dd46f17e2e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:24.823090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:24.867411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91357b-e461-4fdb-8de6-bd192c38cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:24.868303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91357b-e461-4fdb-8de6-bd192c38cd58 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.868581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91357b-e461-4fdb-8de6-bd192c38cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.869150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91357b-e461-4fdb-8de6-bd192c38cd58 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.871249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91357b-e461-4fdb-8de6-bd192c38cd58 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:24.873069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91357b-e461-4fdb-8de6-bd192c38cd58 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:24.875068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 89/177] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:24 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:24.876186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:24.877218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:24.877757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:24.897154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:24.898294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:24.944784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:24.977005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 89/178] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:24.978144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:24.997453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 90/179] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:24.998350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.096086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:25.132275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f878d89-05ab-4507-b3da-fb65d8ce3321 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.132611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad45fc6-17d2-44e2-bbc9-5b8c39ef4f21 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.204441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.205236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.205900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.206178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.206452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.207336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] RBAC: Adding query filter params (role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=d198e1a875754bcc9e77e87149e160fd) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:25.207648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:25.209128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:25.211162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.211677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.212157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] PATH_INFO: `/v3/projects/7c571694986b441d9d725f8df1330686` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.217696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 90/180] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:24 2022] GET /identity/v3/role_assignments?user.id=d198e1a875754bcc9e77e87149e160fd&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:25.219449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] RBAC: Authorizing `identity:get_project(project_id=7c571694986b441d9d725f8df1330686)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:25.221205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:25.223310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 91/181] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:24 2022] GET /identity/v3/projects/7c571694986b441d9d725f8df1330686 => generated 312 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:25.223906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.230623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.313758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:25.424435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b0a75-b06a-4de3-ae05-1a64deaca09c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.449474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6525178-a4ca-46ea-93da-83fe0b31392e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.464102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.464651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.464651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.464950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.465799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=7c571694986b441d9d725f8df1330686, user.id=d198e1a875754bcc9e77e87149e160fd) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:25.466114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:25.469100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9372b-fc2b-43ac-b44a-06d6efb140cd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:25.474106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 92/182] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:25 2022] GET /identity/v3/role_assignments?user.id=d198e1a875754bcc9e77e87149e160fd&scope.project.id=7c571694986b441d9d725f8df1330686&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:25.475500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08586b3-784f-4864-a37e-aa87876b11a4 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.475790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08586b3-784f-4864-a37e-aa87876b11a4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.476068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08586b3-784f-4864-a37e-aa87876b11a4 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.477321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 93/183] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:25.483376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.483696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.483973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.501470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.503091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.503579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.504072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.518435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:25.527003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:25.527927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:25.528725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:25.529327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:25.529327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:25.531468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 91/184] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:25.539636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.541275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:25.662090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74fec6-facc-4951-acf1-703c44293960 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.667123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:25.703489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.704612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.705106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.705593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.706589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:25.707098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:25.708740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:25.719063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 92/185] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:25.725125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.734986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 94/186] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:25.771705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.850469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d1b7e-069d-4eb0-ab2b-c5247f208cd4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.899179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dc245-103f-4423-b46b-668583be0c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.900577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dc245-103f-4423-b46b-668583be0c6a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.901023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dc245-103f-4423-b46b-668583be0c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.901737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dc245-103f-4423-b46b-668583be0c6a None admin] PATH_INFO: `/v3/users/a9ed86ea99fc4faaa721ac5da51c1cbc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.903651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dc245-103f-4423-b46b-668583be0c6a None admin] RBAC: Authorizing `identity:get_user(user_id=a9ed86ea99fc4faaa721ac5da51c1cbc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:25.904430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c18fc-9a77-4565-8f5d-76e1870f480b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:25.905435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dc245-103f-4423-b46b-668583be0c6a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:25.908661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 93/187] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:25 2022] GET /identity/v3/users/a9ed86ea99fc4faaa721ac5da51c1cbc => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:25.910199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8dfb0-18f2-4f2a-b055-c0c8cd685510 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.910663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8dfb0-18f2-4f2a-b055-c0c8cd685510 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.911088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8dfb0-18f2-4f2a-b055-c0c8cd685510 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.912490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 94/188] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:25.913934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8dfb0-18f2-4f2a-b055-c0c8cd685510 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:25.954349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48692b21-23b7-4a0d-9626-e65475aa958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:25.955203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48692b21-23b7-4a0d-9626-e65475aa958f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.955480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48692b21-23b7-4a0d-9626-e65475aa958f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.955750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48692b21-23b7-4a0d-9626-e65475aa958f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.957770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:25.967517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48692b21-23b7-4a0d-9626-e65475aa958f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-48692b21-23b7-4a0d-9626-e65475aa958f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:25.975406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:25.976279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:25.977077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 95/189] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:25.977750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:25.977750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:25.978468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:25.983438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:25.997477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:25.998699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:26.039880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8dfb0-18f2-4f2a-b055-c0c8cd685510 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.067648 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 96/190] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:26.068343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.090900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.091856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.092145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.092419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.168294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.170225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.171639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8832c6-6199-4760-be1a-45be7cadccfb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.202821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.205542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.208931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.210052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.210331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.210655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.211420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:26.211734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.213360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.222116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 97/191] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:26.227726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.274603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 95/192] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:26.275606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.329046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b780de-75fe-4ca4-b43b-ee4f09e0d2fc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.362963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cec0-d7a8-4053-a03b-ca0cf65ea4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.363779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cec0-d7a8-4053-a03b-ca0cf65ea4eb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.364067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cec0-d7a8-4053-a03b-ca0cf65ea4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.364334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cec0-d7a8-4053-a03b-ca0cf65ea4eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.375851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2cec0-d7a8-4053-a03b-ca0cf65ea4eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1cd2cec0-d7a8-4053-a03b-ca0cf65ea4eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:26.383747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:26.384738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:26.385937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 98/193] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:26.387827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124ef0-205c-4382-b60c-c001b2d723fb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.387827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124ef0-205c-4382-b60c-c001b2d723fb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.387827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124ef0-205c-4382-b60c-c001b2d723fb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.388325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 99/194] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:26.388792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d124ef0-205c-4382-b60c-c001b2d723fb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.400910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6116f0-4449-4fc9-9199-9ddbfe5ad812 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.432375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.437932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.438854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.439137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.439408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] PATH_INFO: `/v3/users/7a02c38726c2436ca5c8592d93b9c528` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.446040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.452971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] RBAC: Authorizing `identity:get_user(user_id=7a02c38726c2436ca5c8592d93b9c528)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.454471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.456264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 96/195] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/users/7a02c38726c2436ca5c8592d93b9c528 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:26.459437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.492068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d124ef0-205c-4382-b60c-c001b2d723fb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.526033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa4211-f4b0-48eb-b0bc-bf65d3686629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.526934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa4211-f4b0-48eb-b0bc-bf65d3686629 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.527220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa4211-f4b0-48eb-b0bc-bf65d3686629 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.527495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa4211-f4b0-48eb-b0bc-bf65d3686629 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.528694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa4211-f4b0-48eb-b0bc-bf65d3686629 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.530168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa4211-f4b0-48eb-b0bc-bf65d3686629 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.531896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 100/196] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:26.533029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.533317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.533593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.549839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:26.551075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:26.560678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ad0bb-cc71-4144-abf2-bbef5199b5ab None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.594465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.595162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.595449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.595726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.596488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:26.596815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.598199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.606812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 97/197] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:26.607632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.630170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 101/198] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:26.630170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.671544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.676373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.725211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2acaf6-7387-4455-8414-cda1dee92d49 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.734743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f59410-a989-4422-ad08-992acf9f858e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.763880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.764542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.764823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.765092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.765936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] RBAC: Adding query filter params (role.id=dd25282bc3de4eb296ccdc93ac09b781, scope.project.id=98d7c450055241099126ea71cabc3977, user.id=a9ed86ea99fc4faaa721ac5da51c1cbc) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:26.766241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.767741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.770286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.771149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.771567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.771851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.773705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 102/199] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/role_assignments?user.id=a9ed86ea99fc4faaa721ac5da51c1cbc&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=dd25282bc3de4eb296ccdc93ac09b781 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:26.775618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.857224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.858767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.910493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3384b8e-9e42-4ef4-8de5-bb8e136dd8ba None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:26.917117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.930651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:26.953113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:26.954012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:26.954289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:26.954598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] PATH_INFO: `/v3/users/7a02c38726c2436ca5c8592d93b9c528` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:26.955812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] RBAC: Authorizing `identity:get_user(user_id=7a02c38726c2436ca5c8592d93b9c528)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:26.957269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:26.958962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 103/200] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/users/7a02c38726c2436ca5c8592d93b9c528 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:26.959809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:26.976011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 98/201] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:26.976505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:27.075650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dedf8-f51d-414b-82ce-b5ef05ac042f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.080946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506b6d9-2a0f-45a7-8906-99ddc67b0126 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.112422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.113255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.113516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.113841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.115130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.115296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.116008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.116288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.116555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.116684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.118807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 104/202] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:27.124112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:27.141639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:27.179346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:27.194386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.195985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.231670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb085cb-0a1c-4059-97d4-6ac832ec5c75 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.265908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.266816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.267094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.267368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.268579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.270045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.271965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 105/203] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:27 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:27.272618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:27.301941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 99/204] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:27.302851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:27.353572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:27.378127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8e838-1901-4d07-b808-dea5c3cba294 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.397734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:27.402539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8aa2b-c362-4fff-a77d-65c6c29bb3a5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.412971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.413617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.413897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.414204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.414959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.416564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.446552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 106/205] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:27 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:27.446552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:27.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.447901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=7a02c38726c2436ca5c8592d93b9c528) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:27.447901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.447901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.455671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 100/206] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:27 2022] GET /identity/v3/role_assignments?user.id=7a02c38726c2436ca5c8592d93b9c528&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:27.571541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97bda72-d328-47f7-999e-74e98f4e9c5c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.610884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d55b976-6475-4ca3-af29-06414410c11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.611875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55b976-6475-4ca3-af29-06414410c11d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.612156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55b976-6475-4ca3-af29-06414410c11d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.612413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d55b976-6475-4ca3-af29-06414410c11d None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.613666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55b976-6475-4ca3-af29-06414410c11d None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.615181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d55b976-6475-4ca3-af29-06414410c11d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.617222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 107/207] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:27 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:27.623603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:27.733131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c3a1c-cc6c-497b-b27f-424afc70a98f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:27.751813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:27.772529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:27.773212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:27.773495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:27.773774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:27.775050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] RBAC: Adding query filter params (role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=7a02c38726c2436ca5c8592d93b9c528) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:27.775381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:27.776913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09aad8d-3cb2-4738-a2a6-8a4c355f953b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:27.783227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 101/208] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:27 2022] GET /identity/v3/role_assignments?user.id=7a02c38726c2436ca5c8592d93b9c528&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:29.471513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca1d4b-c347-4b14-b5f9-1098efa05a2c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.472047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca1d4b-c347-4b14-b5f9-1098efa05a2c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.472047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca1d4b-c347-4b14-b5f9-1098efa05a2c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.473700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 108/209] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:29.479886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6026662-03d9-424a-8a8a-3f5074e2d137 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.480321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6026662-03d9-424a-8a8a-3f5074e2d137 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.480429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6026662-03d9-424a-8a8a-3f5074e2d137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.498396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a6026662-03d9-424a-8a8a-3f5074e2d137 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:29.499644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a6026662-03d9-424a-8a8a-3f5074e2d137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:29.574662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 102/210] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:29.590382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca1d4b-c347-4b14-b5f9-1098efa05a2c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:29.641471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:29.673125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c72d20-4d73-4a93-9f1a-55ea8d9304ea None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.673750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c72d20-4d73-4a93-9f1a-55ea8d9304ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.674218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c72d20-4d73-4a93-9f1a-55ea8d9304ea None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.676106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 103/211] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:29.681546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19183c-457a-4aad-917f-8d890965f800 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.682255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19183c-457a-4aad-917f-8d890965f800 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.682755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19183c-457a-4aad-917f-8d890965f800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.700316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0e19183c-457a-4aad-917f-8d890965f800 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:29.701313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca1d4b-c347-4b14-b5f9-1098efa05a2c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:29.702124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0e19183c-457a-4aad-917f-8d890965f800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:29.743259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6719119-9904-4260-bdad-ba8cbf733e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:29.744572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6719119-9904-4260-bdad-ba8cbf733e03 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.745841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6719119-9904-4260-bdad-ba8cbf733e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.746719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6719119-9904-4260-bdad-ba8cbf733e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.760043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6719119-9904-4260-bdad-ba8cbf733e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d6719119-9904-4260-bdad-ba8cbf733e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:29.770332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:29.771362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:29.772383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:29.775229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 109/212] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:29.776688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bcc2c-6c98-4945-ac62-5d04e0a0a8c7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.776975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bcc2c-6c98-4945-ac62-5d04e0a0a8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.777251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bcc2c-6c98-4945-ac62-5d04e0a0a8c7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.778761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 110/213] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:29.783357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bcc2c-6c98-4945-ac62-5d04e0a0a8c7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:29.785072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 104/214] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:29.788646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593635-5325-4cbd-bbac-068bbcd6004b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.788646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593635-5325-4cbd-bbac-068bbcd6004b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.788646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec593635-5325-4cbd-bbac-068bbcd6004b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.812361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ec593635-5325-4cbd-bbac-068bbcd6004b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:29.813549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ec593635-5325-4cbd-bbac-068bbcd6004b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:29.847962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:29.882746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 105/215] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:29.884108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795702-42d2-4304-89c8-9fd6df8daead None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.884397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795702-42d2-4304-89c8-9fd6df8daead None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.884675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795702-42d2-4304-89c8-9fd6df8daead None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.886153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 106/216] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:29.887542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795702-42d2-4304-89c8-9fd6df8daead None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:29.888615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bcc2c-6c98-4945-ac62-5d04e0a0a8c7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:29.895899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:29.924615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:29.925289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.925572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.925845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.926620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:29.926929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:29.928277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cbc89-148f-458e-bff0-1d1fcd9e07b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:29.941516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 111/217] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:29.941516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:29.941516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:29.941516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:29.955900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:29.957131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:30.004587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795702-42d2-4304-89c8-9fd6df8daead None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.025273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 112/218] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:30.026301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.047453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.048363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.048646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.048917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.060752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:30.068555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:30.069667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 107/219] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:30.070905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.102529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.122845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.128861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b328ab-4b2a-41ea-b035-c4a2f34e10a5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.163548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.164352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.164635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.164900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.166223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.167724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.169372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 113/220] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.170234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.175225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3eae5-2fa6-41d0-bc31-9aea0bb7b97e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.210838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.211723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.212069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.212345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] PATH_INFO: `/v3/users/7a02c38726c2436ca5c8592d93b9c528` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.213612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] RBAC: Authorizing `identity:get_user(user_id=7a02c38726c2436ca5c8592d93b9c528)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.215146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.216820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 108/221] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/users/7a02c38726c2436ca5c8592d93b9c528 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.217581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.272367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8fd18-41b7-4d88-a398-f37c54999509 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.305796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.306510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.306822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.307092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.313189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.316881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b76b5e-1cac-484d-a644-03304adaf3eb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.317222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:30.325467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:30.326346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:30.327196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 114/222] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:30.327846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.353566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.354237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.354512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.354815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.355570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:30.355870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.356244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.357273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.365689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 109/223] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.366462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.430706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28287b-05eb-4377-9864-c0b9687724bd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.465501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.466180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.466406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09f658-fcc4-45d7-918a-521c7efe1cf4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.466503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.466763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.467484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.468944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.489570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 115/224] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:30 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:30.490411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.507112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.508011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.508285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.508555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.510269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.511806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.513607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 110/225] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.514390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.547226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.604028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.631267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebf9db-facb-4eda-9bd9-745efcf48b0f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.636167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7d3f5-b06a-4412-813e-0e5911bbbadf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.673963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.675884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.675884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.676050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.676757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:30.677076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.678482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.679576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.681237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.681237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.681237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] PATH_INFO: `/v3/users/3523a028b67b4f609de882971ed4df2e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.690735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 116/226] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.690735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.696177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] RBAC: Authorizing `identity:get_user(user_id=3523a028b67b4f609de882971ed4df2e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.698096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.699906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 111/227] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/users/3523a028b67b4f609de882971ed4df2e => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.700760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.802669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:30.814041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a98b41-e1af-49a7-8830-2b0138dac0a4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.819764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dce1e-d977-4da6-b9e2-6824c8d9f67e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:30.861201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.862673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.863347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.864163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/7a02c38726c2436ca5c8592d93b9c528/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.866943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] RBAC: Authorizing `identity:create_grant(project_id=0d01ad82c82545eab80f18bb23c398f0, user_id=7a02c38726c2436ca5c8592d93b9c528, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.869023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.869188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:30.870200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:30.870616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:30.870984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] PATH_INFO: `/v3/users/7a02c38726c2436ca5c8592d93b9c528` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:30.872703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] RBAC: Authorizing `identity:get_user(user_id=7a02c38726c2436ca5c8592d93b9c528)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:30.874279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:30.876292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 112/228] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/users/7a02c38726c2436ca5c8592d93b9c528 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:30.879246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.889191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 117/229] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:30 2022] PUT /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/7a02c38726c2436ca5c8592d93b9c528/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:30.890429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:30.912747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:31.044965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bab00e-dd6a-4a8c-a253-889c73be521a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:31.089127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:31.092318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:31.092599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:31.092885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:31.095091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:31.096783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:31.098784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 118/230] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:31.104898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:31.115366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d03de-0daf-4431-a621-d49569d1d990 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:31.153608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:31.154386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:31.154762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:31.155113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:31.182452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:31.201892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:31.231062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:31.232764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:31.238689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48bd2a-967d-4269-aefb-f80fa17000b1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:31.275693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:31.276934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:31.277388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:31.277829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/7a02c38726c2436ca5c8592d93b9c528/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:31.280785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d01ad82c82545eab80f18bb23c398f0, user_id=7a02c38726c2436ca5c8592d93b9c528, role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:31.282553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:31.303968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 119/231] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:31 2022] PUT /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/7a02c38726c2436ca5c8592d93b9c528/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:31.350016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 113/232] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:31.356857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:31.488914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3063c5-22e1-444c-af11-e87d1d219a53 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:31.524766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:31.532426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25db92-f7e0-47c9-a753-446bf6af734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:31.533553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25db92-f7e0-47c9-a753-446bf6af734d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:31.534015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25db92-f7e0-47c9-a753-446bf6af734d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:31.534659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25db92-f7e0-47c9-a753-446bf6af734d None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:31.536098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25db92-f7e0-47c9-a753-446bf6af734d None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:31.537874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25db92-f7e0-47c9-a753-446bf6af734d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:31.542672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 120/233] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:31 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:31.546424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:31.623813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:31.677647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498eae95-7213-4d02-86e9-00547f4bf4e8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:31.713487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:31.714497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:31.715013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:31.715452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:31.716480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=3523a028b67b4f609de882971ed4df2e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:31.716958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:31.718596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ec09c-0d19-4e29-80a7-30c1ad257a27 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:31.724552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 114/234] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:31 2022] GET /identity/v3/role_assignments?user.id=3523a028b67b4f609de882971ed4df2e&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:32.822884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f0c21-9977-4011-8a46-5da5055655bd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:32.823426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f0c21-9977-4011-8a46-5da5055655bd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:32.823426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f0c21-9977-4011-8a46-5da5055655bd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:32.825521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 121/235] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:32.831298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f35e-a701-41ab-acbe-dd107fda612d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:32.832210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f35e-a701-41ab-acbe-dd107fda612d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:32.832675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598f35e-a701-41ab-acbe-dd107fda612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:32.859648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d598f35e-a701-41ab-acbe-dd107fda612d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:32.861233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d598f35e-a701-41ab-acbe-dd107fda612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:32.950386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 115/236] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:32.989753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f0c21-9977-4011-8a46-5da5055655bd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.050186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427918-58a9-48e5-af36-420aa713edef None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.050507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427918-58a9-48e5-af36-420aa713edef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.050770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427918-58a9-48e5-af36-420aa713edef None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.052318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 116/237] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:33.057709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.057995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.058269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.095694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:33.096803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:33.136344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f0c21-9977-4011-8a46-5da5055655bd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:33.169338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:33.199638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:33.200450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.200729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.201008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.202374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:33.203871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:33.205533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 122/238] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:33.212848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.213758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 117/239] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:33.259254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.307092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:33.379494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfa189-1107-4325-ac29-862297446eb8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:33.411061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bcfd8-f2cc-4dfb-9687-7f9368dd9456 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:33.445666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:33.446452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.446764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.447033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.447759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:33.449280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:33.472010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fc194-cf3d-47f7-9635-eb30278b6808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:33.472626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 123/240] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:33 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:33.475310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fc194-cf3d-47f7-9635-eb30278b6808 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.475310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fc194-cf3d-47f7-9635-eb30278b6808 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.475310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fc194-cf3d-47f7-9635-eb30278b6808 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.486946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fc194-cf3d-47f7-9635-eb30278b6808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fb0fc194-cf3d-47f7-9635-eb30278b6808 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:33.495731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:33.497058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:33.497874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 118/241] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:33.504850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.527493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba52070-15e9-494d-b27f-4d034d2e90a1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.527815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba52070-15e9-494d-b27f-4d034d2e90a1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.528041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba52070-15e9-494d-b27f-4d034d2e90a1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.529430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 119/242] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:33.534314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.534642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.534920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.552612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:33.554401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:33.612696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:33.690667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 120/243] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:33.699557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406842a-0103-4137-91d6-730702cc9587 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:33.718845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.743941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:33.767495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:33.768569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.769067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.769541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.770559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:33.771072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:33.773573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:33.785035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 124/244] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:33.791783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.860002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949fd16-4156-42c8-ae12-a9dda052f75f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:33.904917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63cf0bc-b026-4d79-a13c-6803598a2384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:33.905980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cf0bc-b026-4d79-a13c-6803598a2384 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.906734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cf0bc-b026-4d79-a13c-6803598a2384 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.907020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cf0bc-b026-4d79-a13c-6803598a2384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.921773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63cf0bc-b026-4d79-a13c-6803598a2384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:33.928177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dcfaa-afa5-4e62-90aa-c9530d859510 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a63cf0bc-b026-4d79-a13c-6803598a2384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:33.930856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:33.931721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:33.932520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:33.933582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 121/245] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:33.934627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ba90e-03fa-40d6-812d-51192227d51e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.934916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ba90e-03fa-40d6-812d-51192227d51e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.935220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ba90e-03fa-40d6-812d-51192227d51e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.936476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 122/246] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:33.941951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ba90e-03fa-40d6-812d-51192227d51e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:33.978637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:33.979213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983d4c5-71cb-43f2-bc99-7d2d8237062c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:33.980318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983d4c5-71cb-43f2-bc99-7d2d8237062c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:33.980550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983d4c5-71cb-43f2-bc99-7d2d8237062c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:33.980864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983d4c5-71cb-43f2-bc99-7d2d8237062c None admin] PATH_INFO: `/v3/users/42047d9f0bf549e7bfbfe7fd0fc411fa` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:33.998364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983d4c5-71cb-43f2-bc99-7d2d8237062c None admin] RBAC: Authorizing `identity:get_user(user_id=42047d9f0bf549e7bfbfe7fd0fc411fa)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.000939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983d4c5-71cb-43f2-bc99-7d2d8237062c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.006627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 125/247] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:33 2022] GET /identity/v3/users/42047d9f0bf549e7bfbfe7fd0fc411fa => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:34.008493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.008790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.009069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.030913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:34.031339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:34.071681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ba90e-03fa-40d6-812d-51192227d51e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.077195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.118055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.119231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.119678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 126/248] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:34.120188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.120359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.121196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.122272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:34.122863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.124512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.135526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 123/249] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:34.141480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.259085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450c480-40a7-46ea-a0a8-478aa215d171 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.311241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.322014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.322792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.324524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.324808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.330784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a599c-e7bc-4ad8-89ab-860e9d4b97fd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.386194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.389222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.390128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.390411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.390744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] PATH_INFO: `/v3/users/7a02c38726c2436ca5c8592d93b9c528` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.392075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] RBAC: Authorizing `identity:get_user(user_id=7a02c38726c2436ca5c8592d93b9c528)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.393594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.398669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 124/250] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/users/7a02c38726c2436ca5c8592d93b9c528 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:34.398669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.443850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.445499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.527945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16f512-de9f-43f1-9cf7-0f4e1e2828cc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.563988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f084a-d844-47a5-99a1-da9772d01e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.564795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f084a-d844-47a5-99a1-da9772d01e48 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.565083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f084a-d844-47a5-99a1-da9772d01e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.565353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f084a-d844-47a5-99a1-da9772d01e48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.582793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 127/251] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:34.582793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.584324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f084a-d844-47a5-99a1-da9772d01e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-758f084a-d844-47a5-99a1-da9772d01e48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:34.592578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:34.593477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:34.594298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 125/252] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:34.595510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d86af-ed60-47c9-b743-7d1cd6047ab8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.595614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d86af-ed60-47c9-b743-7d1cd6047ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.595842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d86af-ed60-47c9-b743-7d1cd6047ab8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.597105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 126/253] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:34.601311 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d86af-ed60-47c9-b743-7d1cd6047ab8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.620096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.682362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.710784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d86af-ed60-47c9-b743-7d1cd6047ab8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.725128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6efb3-f56d-4a77-8b06-59b0f3d0db6e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.753651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.755566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.756242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.756764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.758812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.760852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.763023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 127/254] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:34.763994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.767152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151c60-7f60-4109-b3f2-a66b2c8c9d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.768010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151c60-7f60-4109-b3f2-a66b2c8c9d9c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.768786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151c60-7f60-4109-b3f2-a66b2c8c9d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.769064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151c60-7f60-4109-b3f2-a66b2c8c9d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.842202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15151c60-7f60-4109-b3f2-a66b2c8c9d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.844031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15151c60-7f60-4109-b3f2-a66b2c8c9d9c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.884681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.905935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ffd88-330b-44ba-8bdf-5bd08156562f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:34.917518 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:34.951278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:34.952304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.952794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.952992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 128/255] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:34.953837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.954392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:34.954710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:34.955450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:34.955941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:34.956581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:34.958302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:34.974568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 128/256] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:34.974568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:34.982947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:34.984474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:35.051011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 129/257] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:35.051897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.102928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577727af-1814-4a7d-a04a-b78254872cc8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:35.157440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:35.158783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:35.159290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:35.159766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:35.159955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ac044-1329-401b-8450-9548c1ec0471 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:35.161059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] RBAC: Adding query filter params (role.id=dd25282bc3de4eb296ccdc93ac09b781, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=42047d9f0bf549e7bfbfe7fd0fc411fa) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:35.161628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:35.162145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:35.165673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:35.175068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 129/258] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:34 2022] GET /identity/v3/role_assignments?user.id=42047d9f0bf549e7bfbfe7fd0fc411fa&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=dd25282bc3de4eb296ccdc93ac09b781 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:35.176957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.198609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:35.198928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:35.199816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:35.200107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:35.200360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:35.201778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:35.203447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:35.205853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 130/259] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:35.210556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.311752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275df22-2eae-4c7c-b389-ff45c8c45261 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:35.342545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b31107-c473-4781-bb0b-0e6d13e14228 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:35.384530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:35.385422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:35.385599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:35.385898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:35.386271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:35.386983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:35.387402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:35.387811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] PATH_INFO: `/v3/users/7a02c38726c2436ca5c8592d93b9c528` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:35.391739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] RBAC: Authorizing `identity:get_user(user_id=7a02c38726c2436ca5c8592d93b9c528)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:35.393427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:35.395155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 130/260] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/users/7a02c38726c2436ca5c8592d93b9c528 => generated 283 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:35.399754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.404613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:35.412783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:35.413774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:35.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 131/261] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:35.416962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.451692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:35.527237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1db14-43d6-45f5-93e0-702d8fec93b7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:35.565779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:35.566586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:35.566947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:35.567299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:35.595887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:35.631009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a17950-9900-4eeb-8bc2-4dfcb03694d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:35.667504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:35.669391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:35.696347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:35.697072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:35.697361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:35.697640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:35.698497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=7a02c38726c2436ca5c8592d93b9c528) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:35.698844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:35.700332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:35.714430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 131/262] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/role_assignments?user.id=7a02c38726c2436ca5c8592d93b9c528&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:35.715315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.740137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:35.902093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 132/263] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:35.908979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:35.970764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3619a02-f044-4a29-8af7-a6d3d46ec6fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.036236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.037464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.037464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.037629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.038362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:36.038702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.040334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.050116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 132/264] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:36.056357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.072619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.099635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4f700-7682-4e5e-a2d1-2ebc15386bad None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.157990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.187262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.188171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.188457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.188732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.190015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.191523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.192425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81d1a3-a768-44d9-963b-61bccd6e1bba None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.193263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 133/265] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:35 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:36.204536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.233843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b18c5e-8804-47fe-87a6-40d99f342a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.234782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18c5e-8804-47fe-87a6-40d99f342a83 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.235074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18c5e-8804-47fe-87a6-40d99f342a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.235353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b18c5e-8804-47fe-87a6-40d99f342a83 None admin] PATH_INFO: `/v3/users/3523a028b67b4f609de882971ed4df2e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.236930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b18c5e-8804-47fe-87a6-40d99f342a83 None admin] RBAC: Authorizing `identity:get_user(user_id=3523a028b67b4f609de882971ed4df2e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.238273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b18c5e-8804-47fe-87a6-40d99f342a83 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.240034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 133/266] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:36 2022] GET /identity/v3/users/3523a028b67b4f609de882971ed4df2e => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:36.241408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064567-fbe0-4801-b5c2-c73119181009 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.241702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064567-fbe0-4801-b5c2-c73119181009 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.241978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58064567-fbe0-4801-b5c2-c73119181009 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.243943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 134/267] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:36.247867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58064567-fbe0-4801-b5c2-c73119181009 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.340369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2582b4ee-f87b-4146-bd5d-4733c57da50c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.344025 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.360464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58064567-fbe0-4801-b5c2-c73119181009 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.382481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.383375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.383658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.383931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.385387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] RBAC: Adding query filter params (role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=7a02c38726c2436ca5c8592d93b9c528) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:36.385952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.389160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3d394-8932-4d6c-afc2-b95719203145 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.402761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 134/268] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:36 2022] GET /identity/v3/role_assignments?user.id=7a02c38726c2436ca5c8592d93b9c528&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:36.402761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb92ff-9154-498c-a16a-263fe7d65d0a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.402761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb92ff-9154-498c-a16a-263fe7d65d0a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.402761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb92ff-9154-498c-a16a-263fe7d65d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.411541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe037-418b-4fac-8245-bf33cadd76c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.412945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe037-418b-4fac-8245-bf33cadd76c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.413430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe037-418b-4fac-8245-bf33cadd76c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.415026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe037-418b-4fac-8245-bf33cadd76c1 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.417185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe037-418b-4fac-8245-bf33cadd76c1 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.419138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5fe037-418b-4fac-8245-bf33cadd76c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.419997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d2fb92ff-9154-498c-a16a-263fe7d65d0a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:36.421233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d2fb92ff-9154-498c-a16a-263fe7d65d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:36.422111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 135/269] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:36 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:36.424163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426198e-611c-4a94-91cb-10adf17807f2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.425876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426198e-611c-4a94-91cb-10adf17807f2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.426368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426198e-611c-4a94-91cb-10adf17807f2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.428092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 136/270] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:36.429127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426198e-611c-4a94-91cb-10adf17807f2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.503834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.520379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 135/271] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:36.522464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.522772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.523044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.551140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:36.552735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:36.585586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426198e-611c-4a94-91cb-10adf17807f2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.627299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.627936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.628112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.628602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/3523a028b67b4f609de882971ed4df2e/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.630571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d01ad82c82545eab80f18bb23c398f0, user_id=3523a028b67b4f609de882971ed4df2e, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.631872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.640270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 136/272] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:36.641070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.649433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 137/273] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:36 2022] PUT /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/3523a028b67b4f609de882971ed4df2e/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:36.668843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.672258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.762182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc70e3-6663-4339-a968-ea061cd39031 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.787675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.802891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.803721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.804100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.804274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.808892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4a057-332c-4ea2-8004-121b72d391e7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.819238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:36.828607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:36.829563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:36.830352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 137/274] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:36.839675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.851902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:36.853259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:36.853734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:36.854235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:36.855760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:36.857469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:36.859432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 138/275] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:36.865322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:36.935882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:36.963272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c83b1e-868e-44a4-8699-71a190b53f20 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:36.970126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a52ce0-61d6-49e8-ab78-4b96b6e2a77b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:37.003268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:37.004056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.004344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.004624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.004953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:37.005402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:37.005688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:37.005845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.006224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.006583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.007130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:37.007432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:37.009140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:37.021532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 138/276] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:37.022431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:37.039099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 139/277] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:12:36 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:37.084997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:37.156384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21003b38-d1ff-4111-bf2c-0b4d54a11e16 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:37.193988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce05ca0-d436-479f-aa44-d23c0f78ff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:37.194971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05ca0-d436-479f-aa44-d23c0f78ff56 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.195247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05ca0-d436-479f-aa44-d23c0f78ff56 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.195637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05ca0-d436-479f-aa44-d23c0f78ff56 None admin] PATH_INFO: `/v3/users/93243bf994fb4fb1bc0802b2ea17cc18` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.211716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce05ca0-d436-479f-aa44-d23c0f78ff56 None admin] RBAC: Authorizing `identity:get_user(user_id=93243bf994fb4fb1bc0802b2ea17cc18)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:37.213202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce05ca0-d436-479f-aa44-d23c0f78ff56 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:37.215427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 139/278] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:37 2022] GET /identity/v3/users/93243bf994fb4fb1bc0802b2ea17cc18 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:37.223685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:37.337849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb74e38-d3bc-48c0-8e6d-8590f8da0138 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:37.366139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:37.377720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:37.378606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.378904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.379248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.380300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, user.id=93243bf994fb4fb1bc0802b2ea17cc18, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:37.380615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:37.382117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1e69b-e1ca-4f2a-9e4a-e32ca49364db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:37.388883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 140/279] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Jul 15 15:12:37 2022] GET /identity/v3/role_assignments?user.id=93243bf994fb4fb1bc0802b2ea17cc18&scope.system=all&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:37.558364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c629da-accf-495d-8599-9d05fb00cdee None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.558886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c629da-accf-495d-8599-9d05fb00cdee None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.559273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c629da-accf-495d-8599-9d05fb00cdee None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.561050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 140/280] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:37.566743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.567027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.567305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.591604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:37.592734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:37.685019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 141/281] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:37.719095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c629da-accf-495d-8599-9d05fb00cdee None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:37.852801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c629da-accf-495d-8599-9d05fb00cdee None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:37.854724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:37.898572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:37.899151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:37.899721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:37.900095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:37.922380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:37.930143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:37.931037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:37.931837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:37.936445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 141/282] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:37.938037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:38.113636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdde1d-4d39-4209-914b-c7d93e44bc35 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:38.134277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:38.161055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579401ce-5761-4484-b485-292da86788e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:38.161055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579401ce-5761-4484-b485-292da86788e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:38.161055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579401ce-5761-4484-b485-292da86788e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:38.161055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579401ce-5761-4484-b485-292da86788e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:38.161506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579401ce-5761-4484-b485-292da86788e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:38.161506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579401ce-5761-4484-b485-292da86788e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:38.161506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579401ce-5761-4484-b485-292da86788e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:38.169587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 142/283] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:38.175644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:38.325404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6385386-06b5-4a1c-9f1c-57db6807c38c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:38.373303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:38.374122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:38.374402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:38.374705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] PATH_INFO: `/v3/users/42047d9f0bf549e7bfbfe7fd0fc411fa` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:38.375935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] RBAC: Authorizing `identity:get_user(user_id=42047d9f0bf549e7bfbfe7fd0fc411fa)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:38.377392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:38.379390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 142/284] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:38 2022] GET /identity/v3/users/42047d9f0bf549e7bfbfe7fd0fc411fa => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:38.385368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579401ce-5761-4484-b485-292da86788e1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:38.493672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579401ce-5761-4484-b485-292da86788e1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:38.535282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c44db-d229-4dff-a285-450091c2bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:38.536177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c44db-d229-4dff-a285-450091c2bcbc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:38.536459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c44db-d229-4dff-a285-450091c2bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:38.536729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c44db-d229-4dff-a285-450091c2bcbc None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:38.537978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c44db-d229-4dff-a285-450091c2bcbc None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:38.539668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c44db-d229-4dff-a285-450091c2bcbc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:38.542911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 143/285] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:38 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:38.547664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:38.636460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:38.739211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df379234-552d-492d-a552-338cecf8e9e4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:38.792808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d81a5d-c7d0-40ef-bdaa-65b36d81c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:38.793730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d81a5d-c7d0-40ef-bdaa-65b36d81c3dd None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:38.794009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d81a5d-c7d0-40ef-bdaa-65b36d81c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:38.794341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d81a5d-c7d0-40ef-bdaa-65b36d81c3dd None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/42047d9f0bf549e7bfbfe7fd0fc411fa/roles/dd25282bc3de4eb296ccdc93ac09b781` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:38.796423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d81a5d-c7d0-40ef-bdaa-65b36d81c3dd None admin] RBAC: Authorizing `identity:create_grant(project_id=0d01ad82c82545eab80f18bb23c398f0, user_id=42047d9f0bf549e7bfbfe7fd0fc411fa, role_id=dd25282bc3de4eb296ccdc93ac09b781)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:38.797862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d81a5d-c7d0-40ef-bdaa-65b36d81c3dd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:38.813268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 143/286] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:12:38 2022] PUT /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0/users/42047d9f0bf549e7bfbfe7fd0fc411fa/roles/dd25282bc3de4eb296ccdc93ac09b781 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:38.951006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a19bd-eed2-4441-9194-41ffebe72e99 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:38.951573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a19bd-eed2-4441-9194-41ffebe72e99 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:38.952040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a19bd-eed2-4441-9194-41ffebe72e99 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:38.957108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 144/287] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:38.958996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:38.959294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:38.959584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:38.988131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:38.989354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:39.080643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 144/288] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:39.107350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a19bd-eed2-4441-9194-41ffebe72e99 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:39.163511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:39.239824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a19bd-eed2-4441-9194-41ffebe72e99 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:39.292177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:39.293001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.293278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.293552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.309037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:39.316902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:39.317786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:39.319317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 145/289] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:39.327688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:39.370903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:39.500782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4de1de-2393-4982-804c-46a1f2c15fd0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:39.557755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:39.558447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.558765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.559009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.559808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:39.560111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:39.561503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f5f83-f261-4394-885b-f8fef7ebd9f3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:39.571501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 145/290] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:39.578721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:39.713578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef693-d91a-4f39-b240-161fd908aebf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.713936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef693-d91a-4f39-b240-161fd908aebf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.714127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cef693-d91a-4f39-b240-161fd908aebf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.715858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 146/291] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:39.721026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9de0d-b0a9-495d-892e-d640d70a1246 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.721557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9de0d-b0a9-495d-892e-d640d70a1246 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.722031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9de0d-b0a9-495d-892e-d640d70a1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.727817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b8492-1af1-4325-b057-07a9e99b4954 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:39.747036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:39.762344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ffd9de0d-b0a9-495d-892e-d640d70a1246 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:39.763935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ffd9de0d-b0a9-495d-892e-d640d70a1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:39.791896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c3eac-a9ad-47c7-8f71-bf73b3100a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:39.793343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c3eac-a9ad-47c7-8f71-bf73b3100a2b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.795285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c3eac-a9ad-47c7-8f71-bf73b3100a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.796588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c3eac-a9ad-47c7-8f71-bf73b3100a2b None admin] PATH_INFO: `/v3/users/3523a028b67b4f609de882971ed4df2e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.798861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c3eac-a9ad-47c7-8f71-bf73b3100a2b None admin] RBAC: Authorizing `identity:get_user(user_id=3523a028b67b4f609de882971ed4df2e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:39.800729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c3eac-a9ad-47c7-8f71-bf73b3100a2b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:39.810554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 146/292] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:39 2022] GET /identity/v3/users/3523a028b67b4f609de882971ed4df2e => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:39.810554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56590d-e3d7-448b-82a0-d78b0afd23dc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.810554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56590d-e3d7-448b-82a0-d78b0afd23dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.810554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56590d-e3d7-448b-82a0-d78b0afd23dc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.811245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 147/293] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:39.814642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56590d-e3d7-448b-82a0-d78b0afd23dc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:39.877462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 147/294] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:39.879914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.880656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.881548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:39.903130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:39.904342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:39.913974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:39.949148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56590d-e3d7-448b-82a0-d78b0afd23dc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:39.993851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753b77a-5cae-4a7a-b9cc-b0bc39e51518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:39.994205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 148/295] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:39.994946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753b77a-5cae-4a7a-b9cc-b0bc39e51518 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:39.995703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:39.995850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753b77a-5cae-4a7a-b9cc-b0bc39e51518 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:39.996096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753b77a-5cae-4a7a-b9cc-b0bc39e51518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.047678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.071809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753b77a-5cae-4a7a-b9cc-b0bc39e51518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.073348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753b77a-5cae-4a7a-b9cc-b0bc39e51518 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.108338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287005-f3ca-4a27-a85d-7b8826716f99 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.146833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.147680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.147961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.148258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.159007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:40.166866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:40.167752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:40.168549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 149/296] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:40.169190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.173865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.180131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 148/297] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:40.181660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a80e19-bef1-41ce-8e0a-b4017079ce81 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.181953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a80e19-bef1-41ce-8e0a-b4017079ce81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.182379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a80e19-bef1-41ce-8e0a-b4017079ce81 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.183763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 149/298] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:40.187726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a80e19-bef1-41ce-8e0a-b4017079ce81 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.275055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.285459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46ba75-69a6-4f6e-9d4e-cf642f5e69dd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.303242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a80e19-bef1-41ce-8e0a-b4017079ce81 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.323914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.325144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.325428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.325847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.338140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.339050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.339180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:40.339596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.339669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.340412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:40.340717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.342112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70c89c-b441-4323-842e-c16e597f9103 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:40.346917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:40.347782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:40.348625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 150/299] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:40.350342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.356821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 150/300] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.357910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.358195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.358450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.376027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:40.377230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:40.416999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.458557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 151/301] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:40.458557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.461844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c29404-d59f-4f76-a835-3cbaa1a74ced None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.498357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.499253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.499537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.499817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.501000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.502454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.504210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 151/302] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.507488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.532144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.563643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f564bc9-a190-400f-a290-1931ceef8fd3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.603096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.603847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.604131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.604404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.605179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:40.605486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.606912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.614582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b484c8d-d592-4e1a-a922-ea1ec59bc06d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.615710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 152/303] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.616321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.654125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.660835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.661712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.661994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.662265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] PATH_INFO: `/v3/users/4d422ec06d7044acba427aa9c49c1753` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.676105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] RBAC: Authorizing `identity:get_user(user_id=4d422ec06d7044acba427aa9c49c1753)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.677906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.684507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 152/304] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/users/4d422ec06d7044acba427aa9c49c1753 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.684507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.726460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c90b1-4c7c-4443-afc8-bef7f98e3585 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.755697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.761114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.761922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.762199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.762468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.763677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.765143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.771220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 153/305] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.771220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.794352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafe66a-a75b-4235-83d4-263b259bccb4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.838380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.839039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.839321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.839592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.840436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=3523a028b67b4f609de882971ed4df2e) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:40.840745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.842189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.858559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 153/306] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/role_assignments?user.id=3523a028b67b4f609de882971ed4df2e&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.858559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:40.906078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adb415-1879-4667-b3e5-b560eb305280 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:40.947679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:40.979413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:40.980609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:40.981029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:40.981437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] PATH_INFO: `/v3/users/93243bf994fb4fb1bc0802b2ea17cc18` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:40.982947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=93243bf994fb4fb1bc0802b2ea17cc18)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:40.985310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:40.987941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 154/307] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/users/93243bf994fb4fb1bc0802b2ea17cc18 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:40.989045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:41.023569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf26ef-b10a-40cc-9714-ff89ef6c405c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:41.070484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:41.071188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.071471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.071746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.072598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] RBAC: Adding query filter params (role.id=dd25282bc3de4eb296ccdc93ac09b781, user.id=4d422ec06d7044acba427aa9c49c1753, scope.system=all) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:41.072909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:41.074368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:41.078586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:41.080596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 154/308] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Jul 15 15:12:40 2022] GET /identity/v3/role_assignments?user.id=4d422ec06d7044acba427aa9c49c1753&scope.system=all&role.id=dd25282bc3de4eb296ccdc93ac09b781 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:41.082844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:41.124627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03c773d-3f42-40ed-a7a1-372c06e95cf4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:41.168582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39076b7-7899-48a5-8455-d1fba8de4d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:41.169278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39076b7-7899-48a5-8455-d1fba8de4d35 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.169565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39076b7-7899-48a5-8455-d1fba8de4d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.169842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39076b7-7899-48a5-8455-d1fba8de4d35 None admin] PATH_INFO: `/v3/groups` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.170704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39076b7-7899-48a5-8455-d1fba8de4d35 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:41.172265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39076b7-7899-48a5-8455-d1fba8de4d35 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:41.202571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 155/309] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Jul 15 15:12:40 2022] POST /identity/v3/groups => generated 229 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:41.203370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e59137-f0d1-4e88-98be-0d9ebebfd447 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.203694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e59137-f0d1-4e88-98be-0d9ebebfd447 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.203977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e59137-f0d1-4e88-98be-0d9ebebfd447 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.205671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 156/310] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:41.212553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.212880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.213125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.234142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:41.236317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:41.247754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9424-f6fe-4b04-9aae-ce7bd646923b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:41.268962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:41.291460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:41.292476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.292793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.293073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] PATH_INFO: `/v3/system/users/93243bf994fb4fb1bc0802b2ea17cc18/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.294700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=93243bf994fb4fb1bc0802b2ea17cc18, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:41.296110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:41.309184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 155/311] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Jul 15 15:12:41 2022] PUT /identity/v3/system/users/93243bf994fb4fb1bc0802b2ea17cc18/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:41.332839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 157/312] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:41.357566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:41.475700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c423a-4259-481f-8f18-ccdead5f4954 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:41.513474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:41.514640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.515105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.515632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.527439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:41.535723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:41.536650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:41.537455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:41.539684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 156/313] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:41.545740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:41.614792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:41.664787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698950-b948-417f-a872-edff68b5ffa9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:41.700261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:41.701189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.701642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.702059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.702994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:41.703442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:41.704992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:41.717197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 158/314] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:41.722854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:41.766947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:41.833356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf2340a-eba9-4a1b-a9ac-17467bf0e539 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:41.868981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:41.870638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:41.871151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:41.871634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] PATH_INFO: `/v3/users/42047d9f0bf549e7bfbfe7fd0fc411fa` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:41.873077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] RBAC: Authorizing `identity:get_user(user_id=42047d9f0bf549e7bfbfe7fd0fc411fa)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:41.874759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:41.876963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 157/315] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:41 2022] GET /identity/v3/users/42047d9f0bf549e7bfbfe7fd0fc411fa => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:41.883199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:41.992037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dcc51-5698-4843-8b22-136d8e1389cd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:42.015208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:42.029152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:42.029734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:42.030018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:42.030296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:42.101168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:42.102903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:42.202300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 159/316] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:42.208025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:42.311245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518f1fd-d248-4588-8f37-cd9da697023a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:42.322152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:42.347886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb76f51-c391-455f-b21c-8d8d247c821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:42.348817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb76f51-c391-455f-b21c-8d8d247c821a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:42.349103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb76f51-c391-455f-b21c-8d8d247c821a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:42.349382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb76f51-c391-455f-b21c-8d8d247c821a None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:42.350605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb76f51-c391-455f-b21c-8d8d247c821a None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:42.352135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb76f51-c391-455f-b21c-8d8d247c821a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:42.354173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 158/317] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:42 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:42.360150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:42.386935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:42.465330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cd189-3daa-4886-bc2e-0ee2058dfc4b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:42.503608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:42.504284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:42.504568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:42.504841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:42.505691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] RBAC: Adding query filter params (role.id=dd25282bc3de4eb296ccdc93ac09b781, scope.project.id=0d01ad82c82545eab80f18bb23c398f0, user.id=42047d9f0bf549e7bfbfe7fd0fc411fa) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:42.506007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:42.507550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a912b-c8e1-4134-9e63-dc4ac6795ac5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:42.514470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 160/318] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:12:42 2022] GET /identity/v3/role_assignments?user.id=42047d9f0bf549e7bfbfe7fd0fc411fa&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=dd25282bc3de4eb296ccdc93ac09b781 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:43.300262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1741-e09f-48c1-a6f7-6ded50a7d7d3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.300788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1741-e09f-48c1-a6f7-6ded50a7d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.300788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1741-e09f-48c1-a6f7-6ded50a7d7d3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.302390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 159/319] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:43.308185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739aa11-d6dc-4fc7-8dc6-6c38ac6af263 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.308431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739aa11-d6dc-4fc7-8dc6-6c38ac6af263 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.308734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739aa11-d6dc-4fc7-8dc6-6c38ac6af263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.326712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4739aa11-d6dc-4fc7-8dc6-6c38ac6af263 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:43.327904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4739aa11-d6dc-4fc7-8dc6-6c38ac6af263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:43.343728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eacef-b0a4-4c57-879a-9182ae26817b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.344031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eacef-b0a4-4c57-879a-9182ae26817b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.344314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eacef-b0a4-4c57-879a-9182ae26817b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.345848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 160/320] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:43.350616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.350906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.351228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.368392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:43.369581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:43.402080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 161/321] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:43.402080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968360-c3fe-4c35-a077-f37f5e2a7477 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.402080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968360-c3fe-4c35-a077-f37f5e2a7477 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.402080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39968360-c3fe-4c35-a077-f37f5e2a7477 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.402080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 162/322] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:43.402080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.411358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.411358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.428547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:43.430172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:43.440257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:43.440518 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 161/323] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:43.441466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.507534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 163/324] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:43.508773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.550193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcbec6d-a605-4009-afc1-1fa1ff4a521d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:43.558003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:43.586320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:43.587155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.587436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.588017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.597752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:43.605684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:43.606766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 162/325] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:43.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.615644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a85c6-2586-47ed-b7eb-1d77b7101d27 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:43.650545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:43.651358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.651628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.651893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.653088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:43.654552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:43.656340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 164/326] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:43.657034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.671168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:43.710042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad6168-0482-46a1-ab33-8802ab2bf977 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:43.742947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:43.743823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.744151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.744427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.754616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06246229-3550-4b2e-8f49-c989fb62dfe1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:43.757396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:43.766060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:43.767073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:43.767907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 163/327] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:43.768585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.795069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:43.795744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.796023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.796313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.797049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:43.797366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:43.797866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:43.798796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:43.807228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 165/328] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:43.808075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.877496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315abaf2-8b32-41dc-9283-36cbcfc4e213 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:43.881942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:43.906315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf8d77-3b23-4485-9bdd-e4269ea7b8cc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:43.910058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:43.910763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.911048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.911323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] PATH_INFO: `/v3/groups` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.912010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:43.913630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:43.928928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 164/329] 10.222.0.40 () {60 vars in 1134 bytes} [Fri Jul 15 15:12:43 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:12:43.929970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:43.943389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:43.944402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:43.945590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:43.945868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:43.946737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:43.947076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:43.948544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:43.958438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 166/330] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:43.964372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:44.060796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:44.082662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e5485-4f19-4c36-b480-e0aa071d4d2b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:44.083428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf5f37-f59a-4ad5-b817-496becc706db None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:44.121806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:44.123164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:44.123441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:44.123714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] PATH_INFO: `/v3/users/4d422ec06d7044acba427aa9c49c1753` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:44.125827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] RBAC: Authorizing `identity:get_user(user_id=4d422ec06d7044acba427aa9c49c1753)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:44.126725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:44.127375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:44.127609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:44.127894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:44.128160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] PATH_INFO: `/v3/users/93243bf994fb4fb1bc0802b2ea17cc18` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:44.129179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 165/331] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/users/4d422ec06d7044acba427aa9c49c1753 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:44.129379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] RBAC: Authorizing `identity:get_user(user_id=93243bf994fb4fb1bc0802b2ea17cc18)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:44.131055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:44.132598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 167/332] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:43 2022] GET /identity/v3/users/93243bf994fb4fb1bc0802b2ea17cc18 => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:44.134891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:44.138439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:44.143410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:44.267803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ac82a-c80e-4d3c-a957-66d5aa459d36 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:44.270567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d239d-3aac-43df-acc2-b93388d69e39 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:44.304782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b61729-1538-4a53-95a5-aa9c175a38f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:44.305663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61729-1538-4a53-95a5-aa9c175a38f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:44.305944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61729-1538-4a53-95a5-aa9c175a38f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:44.306214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b61729-1538-4a53-95a5-aa9c175a38f9 None admin] PATH_INFO: `/v3/system/users/4d422ec06d7044acba427aa9c49c1753/roles/dd25282bc3de4eb296ccdc93ac09b781` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:44.307293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:44.307752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b61729-1538-4a53-95a5-aa9c175a38f9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4d422ec06d7044acba427aa9c49c1753, role_id=dd25282bc3de4eb296ccdc93ac09b781)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:44.308335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:44.308783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:44.309252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b61729-1538-4a53-95a5-aa9c175a38f9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:44.309574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:44.310616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] RBAC: Adding query filter params (role.id=c175ea449c794ef699b7b13a0e6803d5, user.id=93243bf994fb4fb1bc0802b2ea17cc18, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:44.311144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:44.312972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930ec43-a477-4993-86f9-eedffea8d2fb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:44.321755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 166/333] 10.222.0.40 () {58 vars in 1334 bytes} [Fri Jul 15 15:12:44 2022] PUT /identity/v3/system/users/4d422ec06d7044acba427aa9c49c1753/roles/dd25282bc3de4eb296ccdc93ac09b781 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:12:44.329026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 168/334] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Jul 15 15:12:44 2022] GET /identity/v3/role_assignments?user.id=93243bf994fb4fb1bc0802b2ea17cc18&scope.system=all&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:46.551787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e565f54-08a3-4630-a757-df5c208e5a8d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:46.552836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e565f54-08a3-4630-a757-df5c208e5a8d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:46.553349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e565f54-08a3-4630-a757-df5c208e5a8d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:46.555661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 167/335] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:46.561522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65787-b373-4781-8639-dc2d20fc15d0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:46.562388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65787-b373-4781-8639-dc2d20fc15d0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:46.562885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b65787-b373-4781-8639-dc2d20fc15d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:46.587557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-53b65787-b373-4781-8639-dc2d20fc15d0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:46.589136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-53b65787-b373-4781-8639-dc2d20fc15d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:46.665540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:46.683834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 169/336] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:46.728014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e565f54-08a3-4630-a757-df5c208e5a8d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:46.753614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:46.776517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2202c0-9fde-4e70-9b51-9948d77adf03 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:46.776826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2202c0-9fde-4e70-9b51-9948d77adf03 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:46.777079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2202c0-9fde-4e70-9b51-9948d77adf03 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:46.780685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 170/337] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:46.783837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:46.784126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:46.784407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:46.816475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:46.817833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:46.906005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e565f54-08a3-4630-a757-df5c208e5a8d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:46.967195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 171/338] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:46.984573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:46.985395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:46.985681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:46.985959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:46.992734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.011191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:47.022116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:47.023368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:47.024432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:47.027229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 168/339] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:47.035625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.131087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe72781-dce4-4f0e-b022-9df6f4b50241 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.134904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:47.159471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0bcf1b-b14e-4847-92df-5034a96d7077 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.166168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:47.189283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33395539-4f55-44e8-882e-9a1de75e60cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.190105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395539-4f55-44e8-882e-9a1de75e60cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.190380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395539-4f55-44e8-882e-9a1de75e60cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.190681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33395539-4f55-44e8-882e-9a1de75e60cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.202300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33395539-4f55-44e8-882e-9a1de75e60cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:47.207799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.208492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.208780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.209060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.209826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:47.210275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-33395539-4f55-44e8-882e-9a1de75e60cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:47.211171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:47.212032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 15:12:47.212840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:47.213431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:47.213498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 172/340] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:47.214068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf7e6-881b-496c-98a9-21c36a0dd436 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.214499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf7e6-881b-496c-98a9-21c36a0dd436 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.214913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf7e6-881b-496c-98a9-21c36a0dd436 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.216569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 173/341] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:47.221823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.222356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.222810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.227393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 169/342] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:47.227393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.243308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:47.245069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:47.330052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 174/343] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:47.332034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.372605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53661dde-5d67-4a86-97a0-a6be630458d0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.412986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.414472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.415070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.415594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.416554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:47.417057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:47.418879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6f1f2-85a8-4641-9b32-f3c74f84c1ae None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:47.429672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 170/344] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:47.432305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ae082-2757-411b-9a99-c5fb74bd882d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.432807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ae082-2757-411b-9a99-c5fb74bd882d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.433329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ae082-2757-411b-9a99-c5fb74bd882d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.435278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 171/345] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:47.436299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ae082-2757-411b-9a99-c5fb74bd882d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.444095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:47.454495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c3ee1-ca28-4899-89b8-3c5fdd3fdbd2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.471449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:47.494203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca4da2-8e4c-49b8-9c54-b568a0d2263f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.495468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4da2-8e4c-49b8-9c54-b568a0d2263f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.495918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4da2-8e4c-49b8-9c54-b568a0d2263f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.496329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca4da2-8e4c-49b8-9c54-b568a0d2263f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.563436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ae082-2757-411b-9a99-c5fb74bd882d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.576673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4da2-8e4c-49b8-9c54-b568a0d2263f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:47.579058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca4da2-8e4c-49b8-9c54-b568a0d2263f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:47.603640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.604471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.604754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.605029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.618123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:47.627300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:47.628237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:47.629145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 172/346] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:47.629817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.692013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 175/347] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:47.693539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.693962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.694456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.707078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:47.713064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:47.714512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:47.731980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:47.752554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2348ef-6197-41b1-9977-7e202a9c45ff None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.792315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 176/348] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:47.793040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.793283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.794141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.794428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.794737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] PATH_INFO: `/v3/users/4d422ec06d7044acba427aa9c49c1753` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.798334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] RBAC: Authorizing `identity:get_user(user_id=4d422ec06d7044acba427aa9c49c1753)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:47.799937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:47.801883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 173/349] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/users/4d422ec06d7044acba427aa9c49c1753 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:47.803123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.912388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d6a4d-8c24-4fe8-adf4-4207ef3a8db7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.954572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e8cee-395a-477a-9322-3a4c51a050a4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:47.967258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:47.967950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.968228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.968498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.969274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:47.969604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:47.971044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b361e-2d3c-4558-8d16-9baf4d08b399 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:47.980431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 177/350] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:47.981643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d36f-ad87-4301-b4ce-7c2312bff9c0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:47.981933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d36f-ad87-4301-b4ce-7c2312bff9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:47.982313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d36f-ad87-4301-b4ce-7c2312bff9c0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:47.983825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 178/351] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:47.984520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d36f-ad87-4301-b4ce-7c2312bff9c0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:47.989292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.004040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.009694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.011305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.011590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.011871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.013409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:48.014902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:48.018501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 174/352] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:48.018501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.128435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d36f-ad87-4301-b4ce-7c2312bff9c0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.149110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d120ce4-6275-4079-8330-32777ec6b426 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.178156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.179140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.179463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.179729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.180682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] RBAC: Adding query filter params (role.id=dd25282bc3de4eb296ccdc93ac09b781, user.id=4d422ec06d7044acba427aa9c49c1753, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:48.181176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:48.183473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbbd15e-954e-446e-af7d-dcff5b5e8137 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:48.196576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.197333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.197610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.198052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.200519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 179/353] 10.222.0.40 () {56 vars in 1308 bytes} [Fri Jul 15 15:12:47 2022] GET /identity/v3/role_assignments?user.id=4d422ec06d7044acba427aa9c49c1753&scope.system=all&role.id=dd25282bc3de4eb296ccdc93ac09b781 => generated 510 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:48.202350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.202781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.203155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.214954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:48.224181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:48.224623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:48.225636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:48.226668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:48.227272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:48.227342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 175/354] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:48.228441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.316224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.320881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.338778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 180/355] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:48.340023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.373030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2e71d-291c-4fa6-b6fa-2d894aa0fc2e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.413101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.413822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.414167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.414441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.472215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8331b0-bb3c-4474-8f75-b30da3109047 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.501337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:48.503261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:48.519538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.520416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.520700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.520962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.527685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:48.529318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:48.531049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 181/356] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:48.534903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.579104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.607702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.635474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 176/357] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:48.640727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.661143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62225935-84e9-493d-abfb-d67525450dbd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.702111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.702963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.703252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.703526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.704535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:48.704639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:48.706160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:48.716889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 182/358] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:48.717742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.753270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1479a-bd90-4966-b65e-e7fe6c663d2a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.793911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fb6a9-996c-4741-b788-130aa0d31454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.794799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb6a9-996c-4741-b788-130aa0d31454 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.795085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb6a9-996c-4741-b788-130aa0d31454 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.795399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb6a9-996c-4741-b788-130aa0d31454 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.809420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fb6a9-996c-4741-b788-130aa0d31454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-576fb6a9-996c-4741-b788-130aa0d31454 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:48.818861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:48.820398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:48.821356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 177/359] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:48.826845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49804c-3603-4736-931d-0d683b99b797 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.827129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49804c-3603-4736-931d-0d683b99b797 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.827402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc49804c-3603-4736-931d-0d683b99b797 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.829400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 178/360] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:48.830175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49804c-3603-4736-931d-0d683b99b797 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.846247 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5d71b-1b25-4fb3-92a8-2ec544a42007 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:48.853607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.872180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:48.898322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:48.898984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:48.899264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:48.899535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:48.900374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=98d7c450055241099126ea71cabc3977) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:48.900683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:48.902153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:48.909777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 183/361] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:48.910758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:48.982062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc49804c-3603-4736-931d-0d683b99b797 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.035873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.036905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.037585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.037585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.039095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:49.040720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:49.042856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 179/362] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:49.043772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.072525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4f59a-93ee-43db-9c45-cbd2cb86d765 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.115434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.116170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.116524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.116795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.165434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:49.182957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:49.188397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d64a7d-dda0-43bf-9824-bed41ea9b1a7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.209539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:49.211665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:49.234398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.235185 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.235517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.235826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.236700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:49.237526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:49.239456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7b90b-802d-4312-b037-919768ba4c79 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:49.251778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 180/363] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:49.253223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.253561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.253886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.275649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:49.276973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:49.331128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 184/364] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:49.337991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.377289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 181/365] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:49.378426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.441278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:49.469224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc265-eb62-4a83-8da2-45127a7caa61 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.494833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:49.517146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.518308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.518623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.518898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.520108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:49.521587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:49.527110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 185/366] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:49.527110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.527352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49637b3d-b85f-4dbb-a39c-5bcbf7147b11 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.564514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.566022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.566321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.566624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.654020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d5ad2-8c2a-479c-a4dd-c566ecb11560 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.666160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:49.668805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:49.706331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.707399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.707748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.707961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.709183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:49.711008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:49.713113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 186/367] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:49.713982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.745768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:49.781608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:49.810630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 182/368] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:49.811858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.864100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d0b22-f275-4ff0-9acd-e643e71b5d49 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:49.913783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:49.913783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:49.913783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:49.913783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:49.927243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:49.940867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:49.941887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:49.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 187/369] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:49.948167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:49.957665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89cd81-a016-446d-87f6-cdb1d6293f50 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.004802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.005515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.005805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.006083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.007990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=0d01ad82c82545eab80f18bb23c398f0) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:50.007990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.008784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.018140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 183/370] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:50.019094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:50.052864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:50.076912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:50.084429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc152a4-6a5b-41f7-8fa3-be2e76c2dd26 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.142864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.143753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.144038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.144317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.145795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.147716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.149719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 188/371] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:49 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:50.150639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:50.174785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9dda9-50bb-499a-b299-a3c918306a89 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.235582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.236026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.236312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.236596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.237999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.239610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.241906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 184/372] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:50.243533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:50.287398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01bd8b-727c-4fa3-82bb-8705e5765a5a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.331353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.332019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.332298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.332569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.333329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:50.333635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.335490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.347778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 189/373] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:50.348699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:50.361248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:50.391675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d661c-f5e2-4799-ba50-a94c1a5d4b68 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.404812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:50.431907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.432636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.432840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.433116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.433979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=98d7c450055241099126ea71cabc3977) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:50.434351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.438560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.445260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 185/374] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:50.447026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:50.482984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a492595-b019-46f2-822c-58b98be7dfdd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.528576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.529480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.529762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.530086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.531429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.532954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.535059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 190/375] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:50.543348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:50.624571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df6416-86e7-455d-8ad8-58a67a06e72b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.640097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:50.668925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745c398-c4a5-4444-b7fa-19bdafc26641 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:50.706556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.707259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.707539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.707819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.709721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:50.710634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:50.711123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:50.711598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:50.712705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=0d01ad82c82545eab80f18bb23c398f0) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:50.713212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.714940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.721536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 191/376] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:50.768258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:50.818279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:50.819927 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:50.957293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 186/377] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:50.962995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:51.075731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1041-55ea-4ba9-97d9-e7921f67f8ea None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:51.112034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff38ddb-c65a-4131-a456-8638a00337d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:51.113178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38ddb-c65a-4131-a456-8638a00337d2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.113606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38ddb-c65a-4131-a456-8638a00337d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.114017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff38ddb-c65a-4131-a456-8638a00337d2 None admin] PATH_INFO: `/v3/projects/7405d44ad14342729b633b56109fe767` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.115402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38ddb-c65a-4131-a456-8638a00337d2 None admin] RBAC: Authorizing `identity:get_project(project_id=7405d44ad14342729b633b56109fe767)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:51.117264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff38ddb-c65a-4131-a456-8638a00337d2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:51.120038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 192/378] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:50 2022] GET /identity/v3/projects/7405d44ad14342729b633b56109fe767 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:51.126303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:51.214201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:51.256277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88f956-70df-4ba8-95da-f5ff9f08ba6f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:51.306365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272d1a4-096f-441a-84bb-1a1efd2d6707 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.306936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272d1a4-096f-441a-84bb-1a1efd2d6707 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.307369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272d1a4-096f-441a-84bb-1a1efd2d6707 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.309331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 193/379] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:51.312065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:51.313179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.313599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.314011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] PATH_INFO: `/v3/groups/7cf9312dde0141cf9b730264abfb5b55` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.323362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.323870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.324317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.338327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] RBAC: Authorizing `identity:get_group(group_id=7cf9312dde0141cf9b730264abfb5b55)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:51.340025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:51.343992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 187/380] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:51 2022] GET /identity/v3/groups/7cf9312dde0141cf9b730264abfb5b55 => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:51.349104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:51.352796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:51.353243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:51.358950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:51.467068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a219cd4-941a-4124-bb30-2bd4ce8a1ff7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:51.480636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 194/381] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:12:51.501700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:51.502361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.502674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.502951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.503787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] RBAC: Adding query filter params (group.id=7cf9312dde0141cf9b730264abfb5b55, role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=7405d44ad14342729b633b56109fe767) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:51.504097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:51.505527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:51.511914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 188/382] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:51 2022] GET /identity/v3/role_assignments?group.id=7cf9312dde0141cf9b730264abfb5b55&scope.project.id=7405d44ad14342729b633b56109fe767&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 260 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:51.516711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:51.671718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609aa821-b785-40af-98a1-e52125acd3cd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:51.711310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:51.712174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.712446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.712709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.721142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:51.728399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:51.739374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:51.740268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:51.741264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 195/383] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:51.751808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:51.754871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:51.886277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caccb7c-819e-44c6-a21d-7a08d71e32a5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:51.921351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:51.922045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:51.922329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:51.922632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:51.923391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:51.923702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:51.925088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:51.934778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 189/384] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:51.947880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:52.094615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f39db5-fdca-4761-8834-f8545ac36ebd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:52.143347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:52.144244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:52.144526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:52.144800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:52.146065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:52.147643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:52.149508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 196/385] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:51 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:52.155443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:52.263563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:52.315896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba92ff6-3614-4975-8cd6-f77cf12842e2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:52.369733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fe501-d1e9-4dbc-8167-1ab7dfc15ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:52.370989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fe501-d1e9-4dbc-8167-1ab7dfc15ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:52.371494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fe501-d1e9-4dbc-8167-1ab7dfc15ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:52.371972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fe501-d1e9-4dbc-8167-1ab7dfc15ff5 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:52.373378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fe501-d1e9-4dbc-8167-1ab7dfc15ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:52.375067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fe501-d1e9-4dbc-8167-1ab7dfc15ff5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:52.378237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 190/386] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:52 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:52.383396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:52.452827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:52.488139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98825f8b-fad2-45b3-b960-198642067a2c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:52.501523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcdfd-7afe-475a-a6ae-ddac7a735d05 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:52.502204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcdfd-7afe-475a-a6ae-ddac7a735d05 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:52.502703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cedcdfd-7afe-475a-a6ae-ddac7a735d05 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:52.504946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 191/387] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:52.514545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:52.514545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:52.514545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:52.532726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:52.534013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:52.553584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:52.554731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:52.555018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:52.555301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:52.559905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7c450055241099126ea71cabc3977, group_id=9256a8e039a54ab5b4f13c56414b7f0b, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:52.561614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:52.592380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 197/388] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Jul 15 15:12:52 2022] PUT /identity/v3/projects/98d7c450055241099126ea71cabc3977/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:12:52.608502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 192/389] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:52.631565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:52.791736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c975-39f1-4c43-90bb-4207c7faee4c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:52.827971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:52.844578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4a1ff-229a-48a3-9da8-eef9604b9f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:52.845407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4a1ff-229a-48a3-9da8-eef9604b9f9b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:52.845691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4a1ff-229a-48a3-9da8-eef9604b9f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:52.845974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4a1ff-229a-48a3-9da8-eef9604b9f9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:52.858391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4a1ff-229a-48a3-9da8-eef9604b9f9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-18c4a1ff-229a-48a3-9da8-eef9604b9f9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:52.867142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:52.868109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:52.869346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 198/390] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:52.875428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:52.954980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:53.010792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0873a8-828a-46a5-9c39-a2922df28187 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.032330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772550d-e405-4ff6-8a96-514dc479da5a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.032657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772550d-e405-4ff6-8a96-514dc479da5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.032891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772550d-e405-4ff6-8a96-514dc479da5a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.034575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 199/391] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:53.045893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.046197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.046466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.058441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.059153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.059434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.059709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.060464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:53.060770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:53.062365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:53.071431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 193/392] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:53.080281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.084334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:53.085618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:53.226576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 200/393] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:53.264339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad06980-1194-442e-8f53-13b5331e8a5f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.273336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.294047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:53.317671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.318596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.318997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.319278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.323342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:53.324818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:53.326601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 194/394] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:53 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:53.332579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.394655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b98bbc-bd2b-42c3-afee-745f657c3211 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.412924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:53.436468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.437849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.439131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.440236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.461689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:53.475280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:53.476215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:53.477060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:53.482656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 201/395] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:53.488969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.536527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b388185-853a-407a-b68b-d07a4f0c7542 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.613170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5026dae-bac9-4bb3-822f-74498aabd8a8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.628784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.629665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.629941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.630207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.635293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:53.636790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:53.638922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 195/396] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:53 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:53.644525 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.658708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.659844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.660320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.660812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.661907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:53.662651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:53.664533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a176c-3065-4a93-899d-dadf25fd308c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:53.674939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 202/397] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:53.676494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c439ca-1153-4bbe-9705-df13a89293f7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.676994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c439ca-1153-4bbe-9705-df13a89293f7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.677484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c439ca-1153-4bbe-9705-df13a89293f7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.679539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 203/398] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:53.680721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c439ca-1153-4bbe-9705-df13a89293f7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.729902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:53.731277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:53.780487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e64eae-8ebb-4a32-a861-b30c7b8c090d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.802304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c439ca-1153-4bbe-9705-df13a89293f7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:53.817372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3a23e-d3ed-426c-8c2f-409f864ca320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.818410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a23e-d3ed-426c-8c2f-409f864ca320 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.818809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a23e-d3ed-426c-8c2f-409f864ca320 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.819165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3a23e-d3ed-426c-8c2f-409f864ca320 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.821526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3a23e-d3ed-426c-8c2f-409f864ca320 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d01ad82c82545eab80f18bb23c398f0, group_id=9256a8e039a54ab5b4f13c56414b7f0b, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:53.823117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3a23e-d3ed-426c-8c2f-409f864ca320 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:53.844685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:53.846035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.846641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 196/399] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Jul 15 15:12:53 2022] PUT /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:12:53.846897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.847451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.848351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c195db-bf96-407a-8438-d53cd620a27a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.848692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c195db-bf96-407a-8438-d53cd620a27a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.848981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c195db-bf96-407a-8438-d53cd620a27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.849656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:53.851626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:53.854045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 204/400] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:53 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:53.862296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:53.878647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-96c195db-bf96-407a-8438-d53cd620a27a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:53.879629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-96c195db-bf96-407a-8438-d53cd620a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:53.974620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 197/401] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:53.974620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e196-b302-42c4-ab6a-6e3f2688ce3e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.974620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e196-b302-42c4-ab6a-6e3f2688ce3e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.974620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9e196-b302-42c4-ab6a-6e3f2688ce3e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:53.974620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 198/402] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:53.984165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:53.984340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:53.984611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.011155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029b2f-bf26-48fe-8c79-066651b811b6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.018449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:54.019807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.022960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:54.052642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.053933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.054386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.054692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.055942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.057394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.059560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 205/403] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:53 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:54.060388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.082394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.112817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 199/404] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:54.114060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.179702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dcd43-ed96-48e1-a22e-144db55680f0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.217102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.217919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.218198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.218566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.229992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:54.235800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275f10-9f4d-454c-a941-913702b6d6a8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:54.238908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:54.239771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:54.240604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 206/405] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:54.241742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.279168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.281904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.282833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.283115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.283386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.285633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] RBAC: Authorizing `identity:create_grant(project_id=98d7c450055241099126ea71cabc3977, group_id=9256a8e039a54ab5b4f13c56414b7f0b, role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.287589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.306620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 200/406] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Jul 15 15:12:54 2022] PUT /identity/v3/projects/98d7c450055241099126ea71cabc3977/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:54.307529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.403599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.412105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbd596-251e-4e13-8fe2-26bc63c035cf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.461690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.462589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.462878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.463156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.492969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:54.502844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:54.503805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:54.504623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 207/407] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:54.510236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.535486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9340b-0d58-4223-bafb-4ea4816fdcab None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.584776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.585472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.585749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.586020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.586801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:54.587108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.588499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.597934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 201/408] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:54.603854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.620842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.628663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7484f4-506e-4a43-ba7f-610e1d672c5d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.665203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.665928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.666208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.666485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.667271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:54.667580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.668989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.678565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 208/409] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:54.684449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.717889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973c410-1756-4f73-b20e-f1efad3908cf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.719462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.755987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.757264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.757745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.758175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.759675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.761316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.763616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 202/410] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:54.770567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.792303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fccdd7-f168-4ea2-93e9-d912548ade6a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.826912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.828517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.828993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.829469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] PATH_INFO: `/v3/groups/7cf9312dde0141cf9b730264abfb5b55` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.830836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] RBAC: Authorizing `identity:get_group(group_id=7cf9312dde0141cf9b730264abfb5b55)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.832494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.834414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 209/411] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/groups/7cf9312dde0141cf9b730264abfb5b55 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:54.840141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:54.866183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:54.949298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b33bbd-2259-4aa2-bed1-7af66f4e4e4b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.986928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:54.987843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:54.988127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:54.988404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] PATH_INFO: `/v3/projects/7405d44ad14342729b633b56109fe767` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:54.990801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4979893-6c24-4375-8679-1e26454beb4f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:54.995328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] RBAC: Authorizing `identity:get_project(project_id=7405d44ad14342729b633b56109fe767)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:54.996835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:54.998955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 210/412] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/projects/7405d44ad14342729b633b56109fe767 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:55.004189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:55.029187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:55.030493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.031016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.031493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.033323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:55.035404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:55.037743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 203/413] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:54 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:55.043591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:55.090725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:55.173439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:55.184433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3601fe-e7d2-4183-aacb-a327b2a1ceb9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:55.202900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e0a3b-af11-41e3-bab2-56b1e15fb275 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:55.234213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:55.235496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.235987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.236471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] PATH_INFO: `/v3/projects/7405d44ad14342729b633b56109fe767/groups/7cf9312dde0141cf9b730264abfb5b55/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.242272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480f5bc-5367-4be0-aad0-6c5ee68a54c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:55.242272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480f5bc-5367-4be0-aad0-6c5ee68a54c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.242272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480f5bc-5367-4be0-aad0-6c5ee68a54c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.242272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480f5bc-5367-4be0-aad0-6c5ee68a54c0 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.242636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480f5bc-5367-4be0-aad0-6c5ee68a54c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d01ad82c82545eab80f18bb23c398f0, group_id=9256a8e039a54ab5b4f13c56414b7f0b, role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:55.244114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480f5bc-5367-4be0-aad0-6c5ee68a54c0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:55.244785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] RBAC: Authorizing `identity:create_grant(project_id=7405d44ad14342729b633b56109fe767, group_id=7cf9312dde0141cf9b730264abfb5b55, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:55.246602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:55.266565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 204/414] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Jul 15 15:12:55 2022] PUT /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0/groups/9256a8e039a54ab5b4f13c56414b7f0b/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:12:55.266565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b666681-d146-44ec-af0a-4a4db5924ff9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.266865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b666681-d146-44ec-af0a-4a4db5924ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.267100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b666681-d146-44ec-af0a-4a4db5924ff9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.268616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 205/415] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:55.269823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 211/416] 10.222.0.40 () {58 vars in 1442 bytes} [Fri Jul 15 15:12:55 2022] PUT /identity/v3/projects/7405d44ad14342729b633b56109fe767/groups/7cf9312dde0141cf9b730264abfb5b55/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:12:55.274633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.274920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.275191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.315168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:55.316295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:55.427599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 206/417] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:55.452167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:55.568985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb1d95-74a0-4763-9eb3-d6c8a415986a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:55.603433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89721-caea-41d8-8053-f21771518794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:55.604579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89721-caea-41d8-8053-f21771518794 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.605042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89721-caea-41d8-8053-f21771518794 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.605481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c89721-caea-41d8-8053-f21771518794 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.616396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c89721-caea-41d8-8053-f21771518794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-16c89721-caea-41d8-8053-f21771518794 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:55.624618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:55.625512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:55.626388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:55.628774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 212/418] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:55.635369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:55.676597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:55.779630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f0e42-b3c9-42df-a564-df7e2d7146da None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:55.832568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:55.833166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:55.833450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:55.833719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:55.834596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:55.834906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:55.836404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:55.850947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 207/419] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:55.856955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89721-caea-41d8-8053-f21771518794 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:55.886954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:55.997333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c89721-caea-41d8-8053-f21771518794 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:56.047412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9168d-0459-40dc-93e7-dfb5cc519894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:56.048073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9168d-0459-40dc-93e7-dfb5cc519894 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.048356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9168d-0459-40dc-93e7-dfb5cc519894 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.048634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9168d-0459-40dc-93e7-dfb5cc519894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.141315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9168d-0459-40dc-93e7-dfb5cc519894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:56.142742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9168d-0459-40dc-93e7-dfb5cc519894 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:56.166805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:56.268803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 213/420] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:56.279949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:56.372957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31784fb3-d0cb-4132-a0b8-9bb676061210 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.373333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31784fb3-d0cb-4132-a0b8-9bb676061210 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.373571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31784fb3-d0cb-4132-a0b8-9bb676061210 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.375322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 214/421] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:12:56.380505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.380805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.381196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.408621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:56.409748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:56.431856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:56.464465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b560e5-ff63-41fd-adb1-446792e13ab5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:56.490889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 215/422] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:56.509672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:56.556117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:56.557033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.557313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.557585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.558829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:56.560305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:56.562089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 208/423] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:56 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:56.566912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:56.569300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:56.669889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275119d-9e7d-4dd6-b905-60a91ac5f8d7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:56.743191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae051ef-9075-43f1-b378-0ae2be7c0642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:56.744829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae051ef-9075-43f1-b378-0ae2be7c0642 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.745350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae051ef-9075-43f1-b378-0ae2be7c0642 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.745882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae051ef-9075-43f1-b378-0ae2be7c0642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.753448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00463ed1-2325-4f59-b54b-7b3181145acb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:56.759995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae051ef-9075-43f1-b378-0ae2be7c0642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dae051ef-9075-43f1-b378-0ae2be7c0642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:56.769426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:56.770433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 15:12:56.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:56.774765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 216/424] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:12:56.776448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61720a6a-fd68-4deb-a66e-a0d65a7f1cbe None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.777032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61720a6a-fd68-4deb-a66e-a0d65a7f1cbe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.777577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61720a6a-fd68-4deb-a66e-a0d65a7f1cbe None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.779488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 217/425] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:56.784567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61720a6a-fd68-4deb-a66e-a0d65a7f1cbe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:56.804111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea137ed-d2e6-4b68-9c2f-149362c79f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:56.805477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea137ed-d2e6-4b68-9c2f-149362c79f51 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.805985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea137ed-d2e6-4b68-9c2f-149362c79f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.806510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea137ed-d2e6-4b68-9c2f-149362c79f51 None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.807987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea137ed-d2e6-4b68-9c2f-149362c79f51 None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:56.809909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea137ed-d2e6-4b68-9c2f-149362c79f51 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:56.812306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 209/426] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:56 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:56.820869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.821560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.822039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.840734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:56.842647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:56.866470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:56.900892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61720a6a-fd68-4deb-a66e-a0d65a7f1cbe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:56.902454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:56.924183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 210/427] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:56.925424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:56.943630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:56.943630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:56.943630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:56.944949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:56.945951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:56.946262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:56.951108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:56.962209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 218/428] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:12:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:56.966201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.056603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cb383-27ba-40fe-a87a-3affdce58c0e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.074007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5afbd-7efa-4f08-9732-b2bc75f3a522 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.097909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.099068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.099906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.100435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.101729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=98d7c450055241099126ea71cabc3977) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:57.102367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.104256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.111426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 211/429] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:56 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:57.112982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.115214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.116792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.116993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.117320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.129796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:57.138614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:57.139701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:57.140544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 219/430] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:57.144198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:57.148332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.198008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:57.252290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a65d0-a68a-46f0-b146-418421e525a9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.294587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.296068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.296551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.297070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.320869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e2570-eef4-4db6-be02-638be2deed1c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.383045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.383752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.384033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.384310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.385114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:57.385481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.387021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.391796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.393716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.398868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 220/431] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:57.407759 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.429746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:57.530587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:57.538776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 212/432] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:57.545969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.553449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d7f75-8c62-4b38-9e01-2fa90d4b7c76 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.592256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.593242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.593691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.594145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.672876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.674897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.682099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f867bd-3876-40ad-ab1c-21eaf8ee3065 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.710998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:57.734236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.735568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.735923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.736270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.737902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.739384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.742247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 213/433] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:57.748805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.783155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:57.809998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 221/434] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:57.811584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.896922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a4a3a-d5c9-4c58-a03c-bab70554c4ad None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.942922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a56316-8eef-415a-8a9b-6fcac88afbd6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:57.946819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f319db3-3dda-43e1-9bdb-733a90aac180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.947820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f319db3-3dda-43e1-9bdb-733a90aac180 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.948113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f319db3-3dda-43e1-9bdb-733a90aac180 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.948726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f319db3-3dda-43e1-9bdb-733a90aac180 None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.950109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f319db3-3dda-43e1-9bdb-733a90aac180 None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.952113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f319db3-3dda-43e1-9bdb-733a90aac180 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.953945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 214/435] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:57.955298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bddf5f-13f5-4559-baf2-3d26c985f4d5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.955590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bddf5f-13f5-4559-baf2-3d26c985f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.956159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bddf5f-13f5-4559-baf2-3d26c985f4d5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.957184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 215/436] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:57.958260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c3654-99da-43ce-87d4-fc56f7720027 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.962947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c3654-99da-43ce-87d4-fc56f7720027 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.963514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c3654-99da-43ce-87d4-fc56f7720027 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.965808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 216/437] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:12:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:12:57.966623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c3654-99da-43ce-87d4-fc56f7720027 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:57.983853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fc1b1-cfa1-4fb0-8c5b-3de2e5738d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:57.984729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc1b1-cfa1-4fb0-8c5b-3de2e5738d78 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.985006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc1b1-cfa1-4fb0-8c5b-3de2e5738d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.985276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fc1b1-cfa1-4fb0-8c5b-3de2e5738d78 None admin] PATH_INFO: `/v3/projects/98d7c450055241099126ea71cabc3977` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:57.986635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fc1b1-cfa1-4fb0-8c5b-3de2e5738d78 None admin] RBAC: Authorizing `identity:get_project(project_id=98d7c450055241099126ea71cabc3977)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:57.988397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fc1b1-cfa1-4fb0-8c5b-3de2e5738d78 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:57.990734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 222/438] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/projects/98d7c450055241099126ea71cabc3977 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:57.992568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d03aa-020c-4f6b-9343-329d3298d01f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:57.992831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d03aa-020c-4f6b-9343-329d3298d01f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:57.993455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d03aa-020c-4f6b-9343-329d3298d01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.010282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-516d03aa-020c-4f6b-9343-329d3298d01f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:58.011551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-516d03aa-020c-4f6b-9343-329d3298d01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:58.027850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.062645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.078678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c3654-99da-43ce-87d4-fc56f7720027 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.081529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 223/439] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:58.094747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.094747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.094747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.101055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:12:58.102342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:12:58.116053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.116723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.117008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.117284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.118423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=c175ea449c794ef699b7b13a0e6803d5, scope.project.id=0d01ad82c82545eab80f18bb23c398f0) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:58.118777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:58.120228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:58.126319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 217/440] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:57 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=c175ea449c794ef699b7b13a0e6803d5 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:58.127229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.185281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 224/441] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:12:58.186185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.263588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab1d3d-9fb7-4f87-9f78-cce319ae81b2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.287426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.305053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.305945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.306205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.306475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.307724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:58.309307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:58.310178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291250f-d7ba-4205-b0de-54f69989c589 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.316476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 218/442] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:58.316476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.334713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.362044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.363410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.363410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.363595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.377554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:58.385739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:58.387074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 225/443] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:58.388098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.440297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b78a10-fc7b-4761-9e57-a424223f054b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.477514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.478330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.478640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.478918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.489943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:12:58.498403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:12:58.499480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 219/444] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:12:58.500494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.504980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7c925-2baf-4872-a05f-3043eeb6cacb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.539180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.539824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.540101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.540368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.541203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=98d7c450055241099126ea71cabc3977) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:58.541509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:58.542997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:58.549068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 226/445] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=98d7c450055241099126ea71cabc3977&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:58.549829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.557854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.604549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.630071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030739bf-66da-47b3-b18c-47aa50e1b134 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.667526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.668210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.668494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.668770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.669536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:58.669841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:58.671285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:58.685692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 220/446] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:58.685891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b1471-9b66-4ae0-8a0b-1d76e5533296 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.686884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.730904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.731807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.732059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.732341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.734592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:58.734592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:58.735434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:58.745146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 227/447] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:12:58.751365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:58.822273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfabb7-36e3-4103-b4c1-0ced6b923596 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.832980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.863354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.863939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.864221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.864494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.874484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:58.878159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0b254-c7fc-4fe8-a9ed-0e4df5e9d6d0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:58.919937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:58.920591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:58.920876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:58.921152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:58.947919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:58.949396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.006403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.008665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.073488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 221/448] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:59.079498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:59.092409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:59.108423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:59.114902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 228/449] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:12:59.118499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:59.192169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59a88c-020a-49c1-86bc-fb0c9931c58f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:59.225746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06e3f0-d225-444a-bf3e-11c98a09d4a6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:59.226267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:59.227162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:59.227440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:59.227710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] PATH_INFO: `/v3/projects/7405d44ad14342729b633b56109fe767` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:59.228891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] RBAC: Authorizing `identity:get_project(project_id=7405d44ad14342729b633b56109fe767)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.230328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.232045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 222/450] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:59 2022] GET /identity/v3/projects/7405d44ad14342729b633b56109fe767 => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:59.237502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:59.260651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:59.261492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:59.261789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:59.262059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] PATH_INFO: `/v3/projects/0d01ad82c82545eab80f18bb23c398f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:59.263225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] RBAC: Authorizing `identity:get_project(project_id=0d01ad82c82545eab80f18bb23c398f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.264684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.266381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 229/451] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:12:59 2022] GET /identity/v3/projects/0d01ad82c82545eab80f18bb23c398f0 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:59.272612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:59.314163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:59.336039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d482f05-4b5c-496d-b3fc-7f73979ee6bf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:59.338701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:59.369575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:59.370356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:59.370665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:59.370915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] PATH_INFO: `/v3/groups/7cf9312dde0141cf9b730264abfb5b55` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:59.372134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] RBAC: Authorizing `identity:get_group(group_id=7cf9312dde0141cf9b730264abfb5b55)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.373034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab256e-7161-4fba-a061-655bbbfb8d92 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:59.373560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.375228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 223/452] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:59 2022] GET /identity/v3/groups/7cf9312dde0141cf9b730264abfb5b55 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:59.381172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:59.408629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:59.409586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:59.409866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:59.410137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] PATH_INFO: `/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:59.411335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] RBAC: Authorizing `identity:get_group(group_id=9256a8e039a54ab5b4f13c56414b7f0b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.412843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.414444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 230/453] 10.222.0.40 () {56 vars in 1180 bytes} [Fri Jul 15 15:12:59 2022] GET /identity/v3/groups/9256a8e039a54ab5b4f13c56414b7f0b => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:59.420118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:12:59.480013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525b81e-138e-4407-979e-c7c379db230d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:59.515435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:59.516103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:59.516395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:59.516675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:59.517910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] RBAC: Adding query filter params (group.id=7cf9312dde0141cf9b730264abfb5b55, role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=7405d44ad14342729b633b56109fe767) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:59.518354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.520352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa4b0f-a648-4924-97be-b8290da4ab64 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.524757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e86f6d-9d9c-4c9c-abe7-ea8ee0af53fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:12:59.530589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 224/454] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:59 2022] GET /identity/v3/role_assignments?group.id=7cf9312dde0141cf9b730264abfb5b55&scope.project.id=7405d44ad14342729b633b56109fe767&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:12:59.567744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:12:59.568465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:12:59.568815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:12:59.569228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:12:59.570088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] RBAC: Adding query filter params (group.id=9256a8e039a54ab5b4f13c56414b7f0b, role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=0d01ad82c82545eab80f18bb23c398f0) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:12:59.570402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:12:59.571929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a01da1-39e0-41aa-8f4c-092f45b9e7ae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:12:59.576011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:12:59.590564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 231/455] 10.222.0.40 () {56 vars in 1376 bytes} [Fri Jul 15 15:12:59 2022] GET /identity/v3/role_assignments?group.id=9256a8e039a54ab5b4f13c56414b7f0b&scope.project.id=0d01ad82c82545eab80f18bb23c398f0&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:02.076478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a146fc2-cbe5-496c-b356-ec97d474a3c4 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.077521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a146fc2-cbe5-496c-b356-ec97d474a3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.078005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a146fc2-cbe5-496c-b356-ec97d474a3c4 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.079902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 225/456] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:02.085165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005bd3-d8be-4494-90af-e433fcca9d5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.086015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005bd3-d8be-4494-90af-e433fcca9d5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.086508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005bd3-d8be-4494-90af-e433fcca9d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.111524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1f005bd3-d8be-4494-90af-e433fcca9d5a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:02.113070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1f005bd3-d8be-4494-90af-e433fcca9d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:02.204712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 232/457] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:13:02.241233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a146fc2-cbe5-496c-b356-ec97d474a3c4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:02.251626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303185b-9565-4795-84c3-121a9ae72dea None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.254228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303185b-9565-4795-84c3-121a9ae72dea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.254744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303185b-9565-4795-84c3-121a9ae72dea None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.258459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:02.258632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 233/458] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:02 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:02.263033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.263322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.263603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.301511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:02.305139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:02.403445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 234/459] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:02.441009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:02.464274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a146fc2-cbe5-496c-b356-ec97d474a3c4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:02.527077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:02.528294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.528857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.529347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.551318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:02.555189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:02.558857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:02.587051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:02.588041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:02.588900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:02.588900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:02.588900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:02.588900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:02.592231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 226/460] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:02.597948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:02.625624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175777d-a075-46ce-bc25-6bc561b17487 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:02.671772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:02.672987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.673487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.673958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.696388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:02.698372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:02.703687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7bca1e6a-8237-4393-95ac-0957bb32a689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:02.725030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:02.725917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:02.726805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:02.726805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:02.726805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:02.726805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:02.729188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 235/461] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:02.731373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bfab-62af-4650-bbda-c5916462b564 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.731882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bfab-62af-4650-bbda-c5916462b564 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.732357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0bfab-62af-4650-bbda-c5916462b564 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.734262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 236/462] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:02.735350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bfab-62af-4650-bbda-c5916462b564 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:02.755657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8df05-963f-42f8-844d-f08efac2f1f4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:02.793095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:02.793828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.794105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.794379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.795165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:02.795466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:02.796905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d2120-9ce5-440e-9d5e-2676b530a90d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:02.809584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 227/463] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:02.815497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.815795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.816081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.833418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:02.834731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:02.854459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0bfab-62af-4650-bbda-c5916462b564 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:02.870763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:02.894030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:02.894726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.895012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.895293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.896596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:02.896902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:02.898408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab818810-ff17-48cf-a082-dd5f64bb7196 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:02.905110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 228/464] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:02.905757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:02.922551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 237/465] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:02.922551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627a530-6703-46bd-bdeb-90730338e9c6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:02.922551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627a530-6703-46bd-bdeb-90730338e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:02.922551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c627a530-6703-46bd-bdeb-90730338e9c6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:02.922551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 238/466] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:02.929307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627a530-6703-46bd-bdeb-90730338e9c6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:02.971508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:03.040595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d670b76-216a-4b3c-8106-5fff3faa851e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.054216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c627a530-6703-46bd-bdeb-90730338e9c6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.082554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27774702-fadf-4586-922b-3d9bc81ae1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.083309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27774702-fadf-4586-922b-3d9bc81ae1ef None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.083569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27774702-fadf-4586-922b-3d9bc81ae1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.083838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27774702-fadf-4586-922b-3d9bc81ae1ef None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.084543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27774702-fadf-4586-922b-3d9bc81ae1ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.085991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27774702-fadf-4586-922b-3d9bc81ae1ef None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.097893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.098731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.100259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.100616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.101498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.103142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.107948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 229/467] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:13:02 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:03.109290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.109588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.109990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.138958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 239/468] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:13:02 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:03.140824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:03.144311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:03.146955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:03.214949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:03.305242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 230/469] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:03.322502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9f0bf-ec24-4ffd-9561-1191aa5a0601 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.339802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:03.366393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.367296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.368016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.368298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.382999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.384560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.390420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:03.405478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:03.406395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:03.407339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:03.407339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:03.407339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:03.407339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:03.410677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 240/470] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:03.414566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:03.472307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad1c05-474c-4d10-b16f-a046d90a0c98 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.507273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:03.516617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.517855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.518338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.518836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.534590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3240781-7723-4e63-bd9a-0c1f098e8bae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.539697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.541186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.564964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.565591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.565877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.566154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.566919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:03.567228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.568666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:03.572891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:03.573719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:03.574619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:03.574619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:03.574619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:03.574619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:03.581583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 241/471] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:03.582741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 231/472] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:03.591407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:03.595426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:03.758192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266b50-92c3-4352-bcbf-585513dbf7bb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.762843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:03.775654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def55067-d8e8-40b5-a40c-334b4fb5a23f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:03.810807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.811565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.811848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.812121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.812951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:03.813350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.814952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb2907b-f7cd-4f37-a713-87989a2531f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:03.815203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ad0fa-af09-4306-88ff-c2a88fb8122e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.816060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2907b-f7cd-4f37-a713-87989a2531f6 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.816818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2907b-f7cd-4f37-a713-87989a2531f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.817299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb2907b-f7cd-4f37-a713-87989a2531f6 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.818936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2907b-f7cd-4f37-a713-87989a2531f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:03.820631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb2907b-f7cd-4f37-a713-87989a2531f6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:03.829500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 232/473] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:03.831002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311a35c-9efb-427b-b9f4-ce5fb4f85134 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.831138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311a35c-9efb-427b-b9f4-ce5fb4f85134 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.831538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311a35c-9efb-427b-b9f4-ce5fb4f85134 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.837347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 233/474] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:03.837347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311a35c-9efb-427b-b9f4-ce5fb4f85134 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:03.838972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:03.857837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 242/475] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:13:03 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:03.860075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:03.860505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:03.860921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:03.890678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:03.892136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:04.028417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 243/476] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:04.036333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311a35c-9efb-427b-b9f4-ce5fb4f85134 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:04.052789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:04.083633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:04.085153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:04.085816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:04.086232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:04.087715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:04.090138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:04.116926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 234/477] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:13:03 2022] POST /identity/v3/users => generated 249 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:04.178607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcea4be-68d0-4fe8-8de4-7f63f3decb1a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:04.210124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:04.221046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:04.222003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:04.222685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:04.223330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:04.224489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:04.226320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:04.235288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.sql.core [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 15:13:04.235288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 15:13:04.235288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [parameters: {'id': '29f28787033942fb90be54d4a12a089a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 15:13:04.235288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 15:13:04.253733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return ref.to_dict() Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:13:04.254865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application yield resource Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise self.value Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application session.commit() Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 15:13:04.256216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.flush() Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application compat.raise_( Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 15:13:04.257243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:13:04.258203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:13:04.259072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [parameters: {'id': '29f28787033942fb90be54d4a12a089a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:04.259861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 15:13:04.260720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:04.263683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 244/478] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Jul 15 15:13:04 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:04.270018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:04.284745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:04.394326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9541e60f-403b-44b2-abd0-97ae08f99dc1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:04.433115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a363d51-984a-49fe-b308-5836ac3de480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:04.433948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a363d51-984a-49fe-b308-5836ac3de480 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:04.434221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a363d51-984a-49fe-b308-5836ac3de480 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:04.434490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a363d51-984a-49fe-b308-5836ac3de480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:04.445749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a363d51-984a-49fe-b308-5836ac3de480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8a363d51-984a-49fe-b308-5836ac3de480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:04.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:04.454800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:04.455742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 235/479] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:04.461190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:04.570179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32efabf-3bbb-44c7-887f-b4529f0cb053 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:04.606413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:04.607560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:04.611738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:04.611738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:04.611738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:04.611738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:04.611738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c538ea-b2bd-4f66-a5bc-97e56308b313 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:04.621685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 245/480] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:05.476421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbb892-eca9-437e-89d8-5c0947b7e03f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:05.476952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbb892-eca9-437e-89d8-5c0947b7e03f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:05.476952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbb892-eca9-437e-89d8-5c0947b7e03f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:05.478575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 236/481] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:05.483948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89423c0a-cd1b-4940-bdb2-47dce1ca6dc1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:05.484337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89423c0a-cd1b-4940-bdb2-47dce1ca6dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:05.484501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89423c0a-cd1b-4940-bdb2-47dce1ca6dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:05.502344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-89423c0a-cd1b-4940-bdb2-47dce1ca6dc1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:05.503586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89423c0a-cd1b-4940-bdb2-47dce1ca6dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:05.564334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:05.585862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 246/482] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:13:05.608871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbb892-eca9-437e-89d8-5c0947b7e03f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:05.679479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:05.731712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbb892-eca9-437e-89d8-5c0947b7e03f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:05.745868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d07d40-71a0-4c11-a259-50abcd06672d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:05.746173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d07d40-71a0-4c11-a259-50abcd06672d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:05.746412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d07d40-71a0-4c11-a259-50abcd06672d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:05.748235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 247/483] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:05.752959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:05.753153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:05.753429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:05.779852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:05.780911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:05.781362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:05.781781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:05.792827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:05.794497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:05.807679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:05.820646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:05.821773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:05.822790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:05.825216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 237/484] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:05.839286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:05.889933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 248/485] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:05.953533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.022019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ac7d9-c93c-4b7d-b348-5b75e204e41e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.070887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:06.080646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.081350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.081633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.081920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.082710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:06.083081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.088236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.108533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 238/486] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:06.114219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.155120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b5141-4412-4bd8-9fab-b65a7c6ac6ed None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.181270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:06.226533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.227365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.227637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.227908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.253681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:06.261966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:06.262928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:06.263728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:06.276318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 249/487] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:06.280869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.320681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9a348-bdf8-4dda-a13a-ff8637b5ea83 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.399362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe0ac3-815a-44e9-a9ea-c510007ccbcc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.403349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.404170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.404451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.404722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.431975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.433544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.443513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.444195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.444473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.444745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.445512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:06.445868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.449087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.465508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 250/488] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:06.467700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:06.468536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:06.469372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:06.469372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:06.469372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:06.469372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:06.469372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 239/489] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:06.474728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.480570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.542971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:06.554872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:06.621795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306b3e4-e99d-4bc8-9773-5489e26cec57 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.627675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c2357-bf7f-434b-9a29-7532baf3b01d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.660499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.661490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.661775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.662052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.662820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:06.663207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.664618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a62b2-80f9-4846-947a-0eaf66187a90 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.672188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.673008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.673288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.673994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.677122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 251/490] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:06.678439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d7d68-acdd-477a-80f4-a28c221bb4b0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.678754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d7d68-acdd-477a-80f4-a28c221bb4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.679034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d7d68-acdd-477a-80f4-a28c221bb4b0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.684358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 252/491] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:06.685182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d7d68-acdd-477a-80f4-a28c221bb4b0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.691785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.693359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f1ae0afa-901a-468c-8157-ee62130f2723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:06.715933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:06.716823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:06.718371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:06.718371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:06.718371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:06.718371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:06.718371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 240/492] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:06.723734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.724146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.724594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.743995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:06.745453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:06.803149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:06.812338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d7d68-acdd-477a-80f4-a28c221bb4b0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.821127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 241/493] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:06.822363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.848953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:06.854785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.855503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.855780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.856051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.864005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.865772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-21bad213-c8c2-4177-9607-93bafb251802 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:06.884032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:06.885076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:06.885998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:06.886510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 253/494] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:06.887813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2b84-9f30-4c51-a0be-f2f5f7c26df9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.888097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2b84-9f30-4c51-a0be-f2f5f7c26df9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.888370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2b84-9f30-4c51-a0be-f2f5f7c26df9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.893828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 254/495] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:06.893828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f2b84-9f30-4c51-a0be-f2f5f7c26df9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:06.932018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04cbdf-791d-4cd0-b666-0050cbc67c48 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:06.971178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:06.971850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:06.972127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:06.972394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:06.973302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:06.973673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:06.974963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:06.986571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 242/496] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:06.987392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.010187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f2b84-9f30-4c51-a0be-f2f5f7c26df9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.046864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb3487-f02f-4477-8884-59ab3dbfa452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.047655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb3487-f02f-4477-8884-59ab3dbfa452 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.047975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb3487-f02f-4477-8884-59ab3dbfa452 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.048262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb3487-f02f-4477-8884-59ab3dbfa452 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.059703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb3487-f02f-4477-8884-59ab3dbfa452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-85fb3487-f02f-4477-8884-59ab3dbfa452 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:07.067369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:07.068305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:07.069106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 255/497] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:07.070256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.070256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.070418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.079038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.087509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:07.088724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:07.098855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ac615-8e7d-4e64-8c6c-edb01a6ba7a2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.099052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.133218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.133898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.134187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.134463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.135340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:07.135654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.137122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.170542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 256/498] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:07.170542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.177641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 243/499] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:13:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:07.178505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.280549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffae5ac-c272-4e64-97de-a0e332418b47 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.282009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39820b51-dbb0-44c4-889e-ef5e722b3239 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.313998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.315029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.315530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.315811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.315908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.316552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.316834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.317113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.317966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:07.318274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.319696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.325463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.326962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.328245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 244/500] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:07.328949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.331602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:07.343571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:07.344669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 257/501] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:07.345803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.351450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.444823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c500c0e-e731-40cb-9c56-eb9a85e890e9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.459974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9e300-6b77-427d-8f2d-f0dbbcd72d05 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.476467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.477124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.477403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.477675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.493830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f653a05-0615-4af3-9770-96601bec072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.494661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f653a05-0615-4af3-9770-96601bec072c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.494946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f653a05-0615-4af3-9770-96601bec072c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.495221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f653a05-0615-4af3-9770-96601bec072c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.505937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f653a05-0615-4af3-9770-96601bec072c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3f653a05-0615-4af3-9770-96601bec072c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:07.513640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:07.514772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:07.515677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 258/502] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:07.516336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866428c4-223a-4728-9d9c-3c360b4c68e9 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.516403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866428c4-223a-4728-9d9c-3c360b4c68e9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.516891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866428c4-223a-4728-9d9c-3c360b4c68e9 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.517957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 259/503] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:07.518711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866428c4-223a-4728-9d9c-3c360b4c68e9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.549771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.551260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.553252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.580765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.619352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866428c4-223a-4728-9d9c-3c360b4c68e9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.649215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 245/504] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:07.650278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.654445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.655294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.655575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.655850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.667638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.669228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:07.687436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:07.688422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:07.689267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:07.689267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:07.689267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:07.689267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:07.689267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 260/505] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:07.689538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.751806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e58d84-6d29-4e78-98ca-65e3e9706e09 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.753325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.787160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.787828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.788102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.788372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.788773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37488b8-6857-4a99-ab62-ba0ee2527945 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:07.789199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:07.789503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.791000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f61787-bdf3-4206-8d1f-eb4ba9836d4f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.804483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:07.807907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 246/506] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:07.809242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.809533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.809808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.823413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:07.824080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:07.824354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:07.824620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:07.825371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:07.825671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:07.826725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:07.827082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:07.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:07.835424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 261/507] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:07.836161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.906728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 247/508] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:07.907922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:07.975235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953200b4-6d65-41c3-b4db-07eb1730802c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.010952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.011760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.012037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.012310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.013888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.024364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6196296-bcb5-4146-b01f-c88c760fd145 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.024539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.025853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:08.043303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:08.044213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:08.045066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:08.045066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:08.045066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.045066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:08.048136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 262/509] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:08.048136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.058778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.059558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.059842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.060114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.060843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:08.061147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.062743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.074509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 248/510] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:08.075342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.084586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.162047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab3a69-d5cd-4976-8aff-8358f349f436 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.181096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e01ccf-f6fb-4564-aa6f-52b43a5b72a3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.195679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.196325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.196613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.196885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.214218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.215237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.215597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.215945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.226366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.227924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:08.245318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:08.246339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:08.247388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:08.247388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:08.247388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.247388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:08.247388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 249/511] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:08.247807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.251535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.264002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.265489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.295463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.349243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731603b2-1bc3-430c-b6f2-2ce2ef7c45a3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.365226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 263/512] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:08.366057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.385683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.386488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.386797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.387068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.398076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.399543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:08.416358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:08.417395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:08.418458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:08.418458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:08.418458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.418458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:08.418458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 250/513] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:08.418811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.468670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e845ec-f74a-4d43-b54c-a258db1aaa87 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.473303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.498717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.505641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.506324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.506612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.506884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.507613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:08.507913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.509325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.518492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58efa5-ab42-40ea-a59e-fb1fa625799e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.520814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 264/514] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:08.521591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.556177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.556974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.557251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.557519 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.564892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.566391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:08.581545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:08.582503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 251/515] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:08.583874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.623435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc691a33-a8a6-48d2-9a39-60bc547ee96b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.657033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.657691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.657969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.658246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.658992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:08.659294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.660681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.673144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 265/516] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:08.673949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.690338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea760a1-018f-4f01-a4c1-e51de3fd3826 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.704703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.727184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.727877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.728394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.728814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.729436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.746348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.747907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:08.765524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:08.766555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:08.767454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:08.767454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:08.767454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:08.767454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:08.767454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 252/517] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:08.767831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.780717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e12cd9-4fcf-4590-bd00-97c9e6fc0468 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.815556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.816201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.816474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.816743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.817628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:08.817946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.819383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.830326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 266/518] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:08.831192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.871628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e57d5-e33e-4358-8fb5-2f1ba68c4c70 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.904801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.905604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.905888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.906164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.912838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.914293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:08.929172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:08.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:08.931355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 253/519] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:08.932246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:08.937656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c989c97-b8b8-4b3f-bf4b-db3af4c26082 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:08.951342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.968507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:08.980564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57120643-a560-4aaf-b490-7d560455484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:08.981244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120643-a560-4aaf-b490-7d560455484a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:08.981531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120643-a560-4aaf-b490-7d560455484a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:08.981807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57120643-a560-4aaf-b490-7d560455484a None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:08.982704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57120643-a560-4aaf-b490-7d560455484a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:08.983017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57120643-a560-4aaf-b490-7d560455484a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:08.984475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57120643-a560-4aaf-b490-7d560455484a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.021950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 267/520] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:09.022810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57120643-a560-4aaf-b490-7d560455484a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.045076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca44cf-2c03-427f-9fb0-78ded5d23d92 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.081112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.081925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.082202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.082493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.090145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.091639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:09.106491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:09.107588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 254/521] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:09.108618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.134208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57120643-a560-4aaf-b490-7d560455484a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.170148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.170850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.171133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.171408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.172143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:09.172454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.173858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.185439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 268/522] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:09.186156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.195754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.212029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee12b7f-573a-4540-a83e-78cb63ac6bf3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.217734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.246709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.247383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.247663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.247923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.248632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.250072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.270007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 255/523] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:13:09 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:09.270886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.299402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f466b-5323-47cf-b444-b49d4c3cd78d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.338371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.339090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.339369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.339638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.340453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:09.340769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.342269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.368935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 269/524] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:09.368935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.391869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66828d-63c1-4a74-83d1-008714d26e57 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.430829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.431769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.432187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.432461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.446701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.473535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.488343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af043c9-8bbe-468e-822a-2e98e4b291ce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.519624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.521239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.524029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.524720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.525004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.525279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.526155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:09.526475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.527979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.548664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 270/525] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:09.549772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.627485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 256/526] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:09.628954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.660288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1ac1a-8ece-4aaf-acba-4c67fb231b0e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.676828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.703060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18937113-1e47-49f7-b671-63f717784adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.704140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18937113-1e47-49f7-b671-63f717784adf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.704428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18937113-1e47-49f7-b671-63f717784adf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.704856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18937113-1e47-49f7-b671-63f717784adf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.714051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18937113-1e47-49f7-b671-63f717784adf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.716233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18937113-1e47-49f7-b671-63f717784adf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-18937113-1e47-49f7-b671-63f717784adf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:09.732168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:09.733132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 271/527] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:09.733967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18937113-1e47-49f7-b671-63f717784adf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.744382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2634f2-09bf-4bab-9895-2e77bbb35fc0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.745480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.786955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.787662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.787901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.788180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.789018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=063c589447364966b64fcc2da5c1aa02) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:09.789331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.790818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:09.796581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 257/528] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/role_assignments?user.id=063c589447364966b64fcc2da5c1aa02&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:09.797376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:09.881601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18937113-1e47-49f7-b671-63f717784adf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.917518 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d249b-8f59-49a4-9855-c2422f2ea2f9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:09.927254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:09.938208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.939380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.939838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.940310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.971191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:09.971910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:09.972194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:09.972469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:09.986155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:09.987662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:10.007954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:10.008894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:10.009743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:10.009743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:10.009743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:10.009743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:10.014660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 258/529] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:10.015846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.033159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.035112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.044604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.120000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05074794-e977-4ae7-957b-f22a77d488df None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.148449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 272/530] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:10.149684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.158802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.159526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.159771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.160052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.160972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:10.161287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.162799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.181538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 259/531] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:10.182309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.212607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.262572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5ae7e-6d89-400e-a8bb-77adebea3b2f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.271138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.304687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.306024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.306447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.306909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.307777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:10.308741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.310588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.321183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28fb38-558f-4809-a10c-dbe31dbc1110 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.326982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 273/532] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:10.328033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.358059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.358665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.359012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.359354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.373683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.375465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:10.404220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:10.405160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:10.406008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:10.406008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:10.406008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:10.406008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:10.409014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 260/533] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:10.411108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.457292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce4d4a-faee-434c-97f2-50518179af83 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.494086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.495047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.495400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.495773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.496707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=9fe7e2ad26c8483e8aecb89225f3b26e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:10.497117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.498714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.510553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 274/534] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/role_assignments?user.id=9fe7e2ad26c8483e8aecb89225f3b26e&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:10.510553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.514363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.568616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.577481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c263f75b-2248-4be9-bc8a-260cc4e40518 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.650338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.651177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.652202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.652501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.668653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.671408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.696919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9681c-aabc-4f87-a2f9-f8a7f6158fb9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:10.699743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:10.700544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:10.701383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:10.702217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 261/535] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:10.706508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.755712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.756383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.756669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.756945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.757685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:10.757992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.760623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.785080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 275/536] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:10.792132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.892781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4546f8-6b8f-4c4f-b955-7c907e8e67b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.905206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.917582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d3d27-6244-4603-a52f-ea6d2555ddb0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:10.955826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.957038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.957627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.958115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.965706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.967560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:10.976077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:10.985646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:10.986661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:10.987508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:10.990095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 276/537] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:10.992547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:10.993230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:10.993512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:10.993807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:10.994776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:10.994977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:10.996002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:10.996356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:11.024254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 262/538] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:11.030350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:11.109655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b6948-ddcf-4b90-8adb-b15c0b3fd04b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:11.177036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:11.177710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.177994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.178268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.179104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:11.179415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:11.180870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:11.205442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 277/539] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:11.213492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d246a-711e-490e-afb8-776ffb89a3b8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:11.215846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:11.236631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:11.290775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:11.291440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.291720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.291994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.292825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=4cfd0067b59048f1850e8eb6e94640ef) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:11.293135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:11.294614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021aa974-8c7f-4cfc-8318-7c76d314e538 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:11.304344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 263/540] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:11 2022] GET /identity/v3/role_assignments?user.id=4cfd0067b59048f1850e8eb6e94640ef&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:11.341229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f72a9-5d89-463d-a3d8-575a519e81b7 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.341548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f72a9-5d89-463d-a3d8-575a519e81b7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.341792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f72a9-5d89-463d-a3d8-575a519e81b7 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.347522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 264/541] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:11.349581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.349939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.350292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.369666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:11.370861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:11.395253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde97b8-e933-4dea-979d-135b5e6c437d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:11.468856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:11.484289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:11.484970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.485255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.485585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.486390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=0d9d155ea97f422490871907f4018532) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:11.486907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:11.488379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:11.490607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 265/542] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:11.497659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 278/543] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:11 2022] GET /identity/v3/role_assignments?user.id=0d9d155ea97f422490871907f4018532&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:11.508290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:11.660827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2dc05-a9c4-4e68-9d83-e902a5d7b3af None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:11.708321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:11.709100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.709344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.709620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.724680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:11.732441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:11.733326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:11.734654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 266/544] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:11.742419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:11.859254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:11.887541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce986b27-830f-434d-8f68-590615c3dd5a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:11.922597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:11.923244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:11.923594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:11.923870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:11.924626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:11.924932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:11.926478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1562f4f-a36e-472c-b837-e7b4c8e945c4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:11.935217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 279/545] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:11.945692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.022822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:12.085440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0152f-17ea-4773-9caf-6365f111796f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.121027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.122203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.122726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.123215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.135023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:12.136764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:12.141280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f70c-060c-4229-82b1-799d6f537256 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.141573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f70c-060c-4229-82b1-799d6f537256 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.141856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f70c-060c-4229-82b1-799d6f537256 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.143515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 280/546] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:12.148729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.149017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.149300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:12.156437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:12.157288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:12.158119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:12.158119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:12.158119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:12.158119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:12.164953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 267/547] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:12.166455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.167198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:12.168407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:12.267126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:12.269001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 281/548] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:13:12.270183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13a0f6-9d66-4acc-826d-27bd1a065076 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.306020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.307268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.307776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.308246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.308348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.309360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:12.309862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:12.311583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:12.329416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 268/549] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:12.330984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.436440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a8592-b4b4-4f6d-adf4-06c381a25da7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.446067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:12.474928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.476219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.476795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.477264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.485335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:12.487156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:12.504615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:12.506095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:12.507204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:12.514658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 269/550] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:12.514825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef967e-7edb-4098-804a-e99154dd7c6d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.516757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.554985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:12.583571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.584401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.584688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.584964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.606378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:12.626743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:12.627735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:12.628559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 282/551] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:12.635777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.636008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1f000-120e-4f79-8cab-baf80e78e260 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.672597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.673495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.673943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.674340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.675413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:12.675819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:12.677716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:12.701623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 270/552] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:12.703787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:12.802892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:12.820270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3578b5-8dec-4383-8be5-0ef8cebe9fb2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.854250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afdee4f-dee2-453d-9ab7-b95236ea4153 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:12.881563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b7d05-53eb-4439-8bb2-e76e0ff521fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.882237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b7d05-53eb-4439-8bb2-e76e0ff521fd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.882571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b7d05-53eb-4439-8bb2-e76e0ff521fd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.891169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:12.891306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b7d05-53eb-4439-8bb2-e76e0ff521fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.900653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:12.901677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:12.902171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:12.902680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:12.903666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:12.904173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:12.905790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:12.915810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 283/553] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:12.921523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.002260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b7d05-53eb-4439-8bb2-e76e0ff521fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.003780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002b7d05-53eb-4439-8bb2-e76e0ff521fd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.058261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9900cb-5b4b-4df1-886f-c4e7a713d1a5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.079266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.101303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.102736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.103225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.103719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.110884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 271/554] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 15:13:13.112292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5df7f-2d42-41cd-a786-4c99fae7b2d5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.112585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5df7f-2d42-41cd-a786-4c99fae7b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.112864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5df7f-2d42-41cd-a786-4c99fae7b2d5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.119038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 272/555] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:13.119915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.120198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.120473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.122547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.124036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.142735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:13.143881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:13.145159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:13.146042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:13.146042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:13.146042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:13.146042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:13.146274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:13.152865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 284/556] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:13.154013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.205528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.227366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 273/557] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 15:13:13.228494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.278800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686bebe-9f65-4bc3-889c-33c2b0aad18b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.316416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.317859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.318351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.318854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.331168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.333328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.344871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc99209-2f07-4fbb-952c-037f0aa26895 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.347261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:13.353994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:13.355395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:13.356298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:13.356298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:13.356298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:13.356298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:13.363187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 285/558] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:13.365290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.384707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.385400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.385685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.385968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.387015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:13.387324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.388894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.403578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 274/559] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:13.408557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.475808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.489439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e20a6a4-138c-45cb-a282-2ce3d14f7acf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.533003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d38448-467f-41a8-961c-f87898085d95 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.538047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.538899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.539180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.539451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.553793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:13.564345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:13.565376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:13.566239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 286/560] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:13.567327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.580141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.581246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.581735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.582245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.583253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:13.583767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.585371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.602855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 275/561] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:13.604060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.631910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.689679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48ed2f-3c16-4384-9f7a-3663a5047c5a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.726024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.727245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8944d-6df7-4b0a-8338-37dafd275342 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.727858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.728339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.728841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.737084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.738880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:13.754209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:13.755522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:13.756492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:13.763934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 287/562] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:13.763934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.772884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.774346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.774896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.775412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.776358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:13.776848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.777770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.780077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.795065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 276/563] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:13.796323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.869795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca7cbf-7aac-4839-a72e-af2bfd132e5b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.892598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:13.903867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.904676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.904955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.905229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.911005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f88ea-57d1-497b-a324-87af3382af29 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:13.912132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.913606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:13.927749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:13.928654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:13.929559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 288/564] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:13.930041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:13.952461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:13.953148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.953436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.953713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.954582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:13.954898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:13.956338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911d2b9-b9a0-4061-900c-f1aa28ba2cac None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:13.974052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 277/565] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:13.975409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15595df-3571-4e9f-96af-b23599db6a13 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:13.975702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15595df-3571-4e9f-96af-b23599db6a13 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:13.975983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15595df-3571-4e9f-96af-b23599db6a13 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:13.978753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 278/566] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:13.978753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15595df-3571-4e9f-96af-b23599db6a13 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.007054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.036285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a94e9b-1652-4de1-9a5a-d9d2e4e80857 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.071524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.072341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.072623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.072897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.079637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15595df-3571-4e9f-96af-b23599db6a13 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.083071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.084546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4dfff5d1-249b-46ee-8540-cafee89783b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:14.101864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:14.102942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:14.104055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:14.104055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:14.104055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:14.104055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:14.104055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 289/567] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:14.104456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153ff2-72fa-428b-9e76-48cfec5021e5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.104550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153ff2-72fa-428b-9e76-48cfec5021e5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.104815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153ff2-72fa-428b-9e76-48cfec5021e5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.110270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 290/568] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:14.111029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91153ff2-72fa-428b-9e76-48cfec5021e5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.117478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.118143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.118431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.118751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.119551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:14.119907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.121302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee275e9-0b9b-4842-ae3d-292b6ebefa25 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.132898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 279/569] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:14.134085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.134404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.134720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.142301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.154370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:14.155614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:14.214460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91153ff2-72fa-428b-9e76-48cfec5021e5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.222281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 280/570] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:14.224443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.243150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.256190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.256991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.257269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.257537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.267580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.269045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-417ca7bd-d4b4-4804-b610-7950f1751950 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:14.287280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:14.288268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:14.289191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:14.289191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:14.289191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:14.289191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:14.289191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 291/571] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:14.289922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.290202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.290472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.306656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:14.307847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:14.331474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fffc5-7452-4e9e-8bbb-05d9f460ccd3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.364992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.365666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.365949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.366227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.367360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:14.367667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.369053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.370538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.374644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 292/572] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:14.375545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.381489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 281/573] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:14.382379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.480889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c05541-8752-4193-a389-f2c6390a1d0f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.487047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d104b7-0106-4624-932b-5d8c18e38492 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.488203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.516446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.517113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.517381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.517657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.518491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=19b3f6ab1b924bb5907bbb33db1a5b4e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:14.518827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.520285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.522280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.523122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.523406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.523681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.525618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 293/574] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/role_assignments?user.id=19b3f6ab1b924bb5907bbb33db1a5b4e&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:14.526421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.537088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:14.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:14.546809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:14.547702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 282/575] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:14.548411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.617607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.652980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e11e7-8f70-41c7-857a-fd3354668f34 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.673349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb14b76-fc5b-48c9-8c35-d392951a7184 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.692579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.693642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.693642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.693970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.694609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:14.694919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.696302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.712149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 294/576] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:14.713466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.719014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.719999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.720402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.720682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.729576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.731132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:14.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:14.747421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 283/577] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:14.748394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.751939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.828795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16715e4-336c-458b-bb53-0af7ac577c94 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.859593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3d016-9a2a-43b0-91eb-982a995d482a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:14.867261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.868184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.868459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.868725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.874310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:14.882164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:14.890900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:14.891955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:14.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 295/578] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:14.893535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:14.899456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:14.900223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:14.900506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:14.900782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:14.901548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:14.901860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:14.903329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:14.916696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 284/579] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:14.917456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.011209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.021612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7259183-2ef8-4d31-a904-415753204b5a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.037292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959883df-0ff6-4512-8b80-1bf94ba12561 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.055578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.056390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.056672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.056951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.063851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.065314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.073209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.073895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.074184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.074462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.075348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:15.075656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.077078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:15.079685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:15.080768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:15.081765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:15.085700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 296/580] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:15.086582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.099613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 285/581] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Jul 15 15:13:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:15.100583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.137177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.201992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b5db5-7d03-4bc0-aa94-c352811943ec None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.208598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133f862-37ab-491d-a98d-147b4513b3da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.240519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.241202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.241854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.242130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.242923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:15.243236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.244623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.246684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.248022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.248479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.248923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.253133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 297/582] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:15.254101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.262767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.264576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.277229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:15.286202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:15.287195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:15.288104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:15.288104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:15.288104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:15.288104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:15.290850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 286/583] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:15.292437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.375693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd33846-8a0b-42ec-a517-c2de060a4d3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.385970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.403138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1edd50-bc86-4f43-aa99-c6252bb0722d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.410756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.411560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.411848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.412127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.412918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:15.413226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.414807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.427570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 298/584] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:15.431824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.438473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.439381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.439663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.439934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.451881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.453399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:15.470391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:15.471310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:15.472253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:15.472253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:15.472253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:15.472253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:15.472253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 287/585] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:15.472578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.516277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.547157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a2b2f-96a5-445d-9ae1-30068dafe9e8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.583392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e508e7-ae2e-490a-9573-86886532e59d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.586438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.587296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.587582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.587860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.599568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.601191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.618318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.619031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.619316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.619591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.620317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:15.620600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:15.621435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:15.622320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:15.622320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:15.622320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:15.622320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:15.622564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.622639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 299/586] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:15.623104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.626335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.633775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.646708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 288/587] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:15.648224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.738885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ff365-dbe2-4caa-942e-43f0af5e2e41 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.753513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b95d35-99ef-4348-8288-db6f80d76bb8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.765846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.779021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.779936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.780204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.780487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/063c589447364966b64fcc2da5c1aa02/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.790342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.791233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.791702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.791979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:15.792783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:15.793090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.794549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.796694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=063c589447364966b64fcc2da5c1aa02, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:15.798128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:15.812203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 289/588] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:15.812203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.818140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 300/589] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:15 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/063c589447364966b64fcc2da5c1aa02/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:15.823500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:15.898793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:15.947575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe681a04-4113-4112-987f-b5e60b23ed63 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.961263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e06d4e-341e-4905-b420-74063a1d6a84 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:15.987787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:15.990720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:15.991251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:15.992837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.001795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.003350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.010666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.011478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.011785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.012033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.012792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:16.013075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.014455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:16.018655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:16.019687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 301/590] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:16.020901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.031288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 290/591] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:16.032252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.063455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.152252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bdf61-8762-479d-bcb2-07d755267a6b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.161779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cc714-48a3-44b6-b5a7-51e6d7e93fe7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.184456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.200243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.202322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.202904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.203455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.205542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.206476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.206784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.207202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.208588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:16.208900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.210477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.214419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.216144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.235524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 291/592] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:16.235524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:16.236895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:16.237782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 302/593] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:16.243396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.358895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cebf20-b95f-4de4-838a-cc292ff1beed None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.367629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.377433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec60cb5-bc83-4fea-a3c8-d54de4c6c47e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.407539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.408769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.409257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.409725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.418122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.419988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.420485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.420795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.421070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.421338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.433773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.435358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:16.437375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:16.438299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:13:16.439299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:16.441896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 292/594] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:16.443019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:16.461074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:16.462014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:16.462943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:16.462943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:16.462943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:16.462943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:16.462943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 303/595] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:16.463212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.487347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.563288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399cac2-9459-49d9-a07a-afd1da2a1e32 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.580925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da68b6-61e7-4e00-8002-32c87287272f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.601511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80430390-819b-4254-90dd-28f48e625414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.602196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80430390-819b-4254-90dd-28f48e625414 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.602609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80430390-819b-4254-90dd-28f48e625414 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.603086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80430390-819b-4254-90dd-28f48e625414 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.604262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80430390-819b-4254-90dd-28f48e625414 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:16.604571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80430390-819b-4254-90dd-28f48e625414 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.605994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80430390-819b-4254-90dd-28f48e625414 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.621592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.622269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.622571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.622848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.623682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:16.629194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 293/596] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:16.629342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.629422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80430390-819b-4254-90dd-28f48e625414 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.629492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966996-947e-447b-9d6b-8262a923ac9f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.647777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 304/597] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:16.649215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff385b89-fd52-4184-8601-8a84ad851d5b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.649501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff385b89-fd52-4184-8601-8a84ad851d5b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.649773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff385b89-fd52-4184-8601-8a84ad851d5b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.655770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 305/598] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:16.656619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff385b89-fd52-4184-8601-8a84ad851d5b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.661653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.735532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.742404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80430390-819b-4254-90dd-28f48e625414 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.772778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff385b89-fd52-4184-8601-8a84ad851d5b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.774938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.775608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.775889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.776162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.776885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:16.777188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.778590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.791109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 294/599] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:16.791908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.816475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a0206-e780-41e9-b28f-5a137315becc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.817890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0206-e780-41e9-b28f-5a137315becc None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.818346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0206-e780-41e9-b28f-5a137315becc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.818950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0206-e780-41e9-b28f-5a137315becc None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/9fe7e2ad26c8483e8aecb89225f3b26e/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.834744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a0206-e780-41e9-b28f-5a137315becc None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=9fe7e2ad26c8483e8aecb89225f3b26e, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.836568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a0206-e780-41e9-b28f-5a137315becc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.860649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 306/600] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:16 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/9fe7e2ad26c8483e8aecb89225f3b26e/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:16.862439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.862751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.863071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.889799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:16.891528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:16.902701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537cb84-23b0-4d05-b48b-2c7dd98072ef None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:16.944826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:16.945799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:16.946130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:16.946403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:16.955641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:16.961163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:16.962711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:16.982406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 307/601] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:16.983745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:16.994759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:16.995755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:16.996685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:16.996685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:16.996685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:16.996685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:16.996685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 295/602] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:17.003396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.028123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.112601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a0a68-83f8-4410-8734-d972b66ab1c8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.135129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41efc1-273f-4663-9c3b-3b986c316305 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.155189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.156004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.156284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.156556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.166749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.169080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.172112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.172859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.173209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.173590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.174395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:17.174836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.176374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:17.185066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:17.185962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:17.186921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:17.187649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 308/603] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:17.190551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.194178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 296/604] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:17.195178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.251587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.273097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.318421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c42fb5-bb41-4b0a-ba4e-1bc91f4055b6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.320702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f12571-1585-4a2d-a277-8b815a258fe0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.354442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.355529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.355955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.356395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.366027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.367016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.367303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.367580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.368652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:17.368962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.369197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:17.370389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:17.377567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:17.378512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:17.379696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:17.386723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 297/606] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:17.386723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.387049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 309/606] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:17.391762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.499853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa732f31-0eb0-420d-94d6-fd6f5ad65332 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.503329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.520655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473adc5-a5f2-40b1-ae9b-d9a6d45d5300 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.539159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.539844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.540174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.540446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.541205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:17.541510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.542927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.555950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.556393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 310/607] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:17.557460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.567498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.569148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.569610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.570103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.580229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.582500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:17.602334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:17.603726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:17.604604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:17.612083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 298/608] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:17.612083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.680026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a6642-1c28-48f4-be5f-4ed00ae2b987 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.715992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.716935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.717210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.717486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/4cfd0067b59048f1850e8eb6e94640ef/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.733103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=4cfd0067b59048f1850e8eb6e94640ef, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.734721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.737133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4ba8e-6d83-4141-92ad-be7c371a7fd1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.758710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 311/609] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:17 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/4cfd0067b59048f1850e8eb6e94640ef/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:17.759826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.786189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.791627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.793093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.793571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.794039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.819941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.821792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:17.854546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:17.855510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:17.856400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:17.856400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:17.856400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:17.856400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:17.861056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 299/610] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:17.865899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:17.888645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5283861-20f2-46ac-b9c1-2d08dc9b2f43 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:17.911665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:17.934496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:17.935326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.935608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.935883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.942584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:17.942584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:17.942584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b7ae4-ce36-42ec-9010-3ddc9f7cd734 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:17.956881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 312/611] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:17.958502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8169-fdaf-467f-9e21-bc63a5404481 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:17.958818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8169-fdaf-467f-9e21-bc63a5404481 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:17.959091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc8169-fdaf-467f-9e21-bc63a5404481 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:17.964449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 313/612] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:17.964449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8169-fdaf-467f-9e21-bc63a5404481 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.025097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780825d6-085f-47ba-85af-04f142c54683 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.054490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:18.071467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.072295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.072679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.073057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.073897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:18.074309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.075822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d30889a-4949-498e-8276-9eef23f249f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.080984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc8169-fdaf-467f-9e21-bc63a5404481 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.089811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 300/613] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:18.091359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.091757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.092139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.110399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:18.111849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:18.125013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.125913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.126195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.126468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/0d9d155ea97f422490871907f4018532/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.142214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=0d9d155ea97f422490871907f4018532, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.143837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.163357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 314/614] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:17 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/0d9d155ea97f422490871907f4018532/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:18.164535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.187620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 301/615] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:18.214166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.275627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:18.367589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47d3b6-3e88-49d6-9856-1a97db45a21d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.386165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02979506-d28c-4830-a04a-5d7d40db713f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.396151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:18.405324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.406217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.406506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.406835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.415636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.417222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.423712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.424990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.425276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.425623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:18.433273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:18.434237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:18.435211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 315/616] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:18.437070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:18.441250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:18.446697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:18.447638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:18.448499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 302/617] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:18.455307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.548169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f543c-cbd7-47c8-b5bb-41d7e7fc6ed0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.553586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:18.563666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a425b-b842-4ed7-97e6-60cb07feba4f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.588168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.589236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.589692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.590104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.591131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:18.591657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.593352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.600802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.601414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.601694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.601965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.602755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:18.603217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.604901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.614340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 303/619] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:18.614581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 316/619] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:18.620522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.620825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.671978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:18.749756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e38bb-b2dd-4df8-bd8f-0be91681fc6d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.786076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.787816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.788104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.788381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.795563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d77f75-e2a1-49f7-b2b7-1a30e6922264 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:18.813634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.815163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:18.844874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:18.846180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:18.847260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:18.847260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:18.847260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:18.847260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:18.847260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 317/620] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:18.852491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.887235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:18.888340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:18.888767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:18.889185 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:18.910303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:18.912042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:18.950548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:18.952009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:18.952929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:18.952929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:18.952929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:18.952929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:18.953761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 304/621] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:18.960229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:18.978860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:18.998686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f7f5a-f199-4078-82af-6e31e582764b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.003093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:19.038270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.038987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.039278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.039562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.040300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:19.040629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.042121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.054659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 318/622] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:19.060595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.176711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5852-9fc2-43af-bd5a-960f755af515 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.213442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.214253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.214551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.214841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.222655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.224271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:19.239861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:19.240849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:19.243183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 319/623] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:19.247869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63563b7c-63fc-4afe-869a-9fcb90fd9428 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.250273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.282962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:19.318705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b309f-d859-4888-9989-f59807d25591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.319413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b309f-d859-4888-9989-f59807d25591 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.319694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b309f-d859-4888-9989-f59807d25591 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.320073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b309f-d859-4888-9989-f59807d25591 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.320815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b309f-d859-4888-9989-f59807d25591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:19.321122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b309f-d859-4888-9989-f59807d25591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.322629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b309f-d859-4888-9989-f59807d25591 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.335562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 305/624] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:19.336841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05f8c1-326f-4f16-9aa0-a1be1f9913e9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.337128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05f8c1-326f-4f16-9aa0-a1be1f9913e9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.337504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05f8c1-326f-4f16-9aa0-a1be1f9913e9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.342942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 306/625] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:19.343756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05f8c1-326f-4f16-9aa0-a1be1f9913e9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.390878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89f08f-2863-4b1c-b56b-9131a077b941 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.416736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:19.435249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.435783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.436071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.436345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.437192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:19.437504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.439559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f1a49-c06c-4a22-a0aa-20796ea09a19 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.452611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 320/626] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:19.453872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.454299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.454548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.466338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05f8c1-326f-4f16-9aa0-a1be1f9913e9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.474142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:19.475788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:19.512887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.514001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.514544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.515015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.525201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.527699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:19.548474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:19.549686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:19.550928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:19.553141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 307/628] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:13:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:19.554188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.554803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 321/628] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:19.559201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.575658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:19.676474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821fde-d644-4d87-9cbd-375c5eabf014 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.701920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:19.703948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0156-c427-4dfd-9c13-aaec6cf760eb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.714913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.716031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.716463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.716882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/19b3f6ab1b924bb5907bbb33db1a5b4e/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.732743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=19b3f6ab1b924bb5907bbb33db1a5b4e, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.734405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.749700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 308/629] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:19 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/19b3f6ab1b924bb5907bbb33db1a5b4e/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:19.750887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.756252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.757163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.757539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.757910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.758941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:19.759349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:19.760948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:19.797857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 322/630] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:13:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:19.804003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:19.864444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873d3f0-f2eb-48be-b449-269e988ca0bb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:19.931340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:19.958956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:19.960920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:19.961383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:19.966462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:19.986281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:19.996180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:19.997419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:19.998571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:20.001129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 309/631] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:20.007829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.029288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a3106-f48e-4d5c-ac52-6e55c87257b5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.047553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.069979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cff31-223c-4f0d-a523-c0887e3d0336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.070590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cff31-223c-4f0d-a523-c0887e3d0336 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.070875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cff31-223c-4f0d-a523-c0887e3d0336 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.071148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cff31-223c-4f0d-a523-c0887e3d0336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.134658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f182f16-17ec-4da6-99ba-d7de2592992b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.152865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cff31-223c-4f0d-a523-c0887e3d0336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.154418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cff31-223c-4f0d-a523-c0887e3d0336 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.182258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.183948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.184453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.185165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.186779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:20.187298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.189275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb03bc-c726-4722-8453-8f358e9771c0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.201322 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 310/632] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:20.203782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab1d29-3c67-4070-bfec-e6fbf8b99f64 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.204569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab1d29-3c67-4070-bfec-e6fbf8b99f64 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.205071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab1d29-3c67-4070-bfec-e6fbf8b99f64 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.207302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 311/633] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:20.212114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab1d29-3c67-4070-bfec-e6fbf8b99f64 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.223731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.272204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 323/634] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 15:13:20.272204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.272204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.272204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.288768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:20.290125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:20.348803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.358596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab1d29-3c67-4070-bfec-e6fbf8b99f64 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.370546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 324/635] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:20.370546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.417572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.418446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.418760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.419620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.433683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.435680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:20.459686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:20.460664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:20.461548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:20.461548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:20.461548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:20.461548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:20.463688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 312/636] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:20.463688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.479105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.489635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a4aaf-9559-43d2-8f7e-d277901cdfe6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.526764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.527889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.528011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.528289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.539299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.540778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:20.559483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:20.560471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:20.561353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:20.561353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:20.561353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:20.561353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:20.561353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 325/637] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:20.561641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.596166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead61d7-1a04-4f58-948e-3a544e11894b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.630618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c90b48-fd2c-48f6-8be1-a05c42932137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.631367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90b48-fd2c-48f6-8be1-a05c42932137 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.631651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90b48-fd2c-48f6-8be1-a05c42932137 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.631926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c90b48-fd2c-48f6-8be1-a05c42932137 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.644262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c90b48-fd2c-48f6-8be1-a05c42932137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-c8c90b48-fd2c-48f6-8be1-a05c42932137 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:20.652317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:20.653303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:20.655028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 313/638] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:20.655757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf41e0-2096-4aed-80eb-7417b96c7976 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.655757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf41e0-2096-4aed-80eb-7417b96c7976 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.655757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf41e0-2096-4aed-80eb-7417b96c7976 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.657050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 314/639] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:20.657589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf41e0-2096-4aed-80eb-7417b96c7976 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.670454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.680450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e6618-dbf8-4bd6-ae10-44bf70b1a329 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.718219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.718949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.719237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.719516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.720239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:20.720551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.721944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.738701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 326/640] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:20.738701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.748196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.767976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf41e0-2096-4aed-80eb-7417b96c7976 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.801855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.802539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.802832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.803110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.803836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:20.804144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.805534 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e626b749-4e84-4fc2-b532-b436a7332e9f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.817200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 315/641] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:20.818414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.818732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.819009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.834859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:20.836059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:20.849623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347bec0-12be-4f69-bf32-23ac5216dabc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:20.884981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:20.885660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:20.885949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:20.886229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:20.887011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:20.887323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:20.888705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:20.897439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 327/642] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:20.898262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.904481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:20.922539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 316/643] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:20.922539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:20.976474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.017227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d6de6-fdcf-4511-8a5a-dab818a01328 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.024076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5184a9-faea-47e5-ada8-b7ae9612f449 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.054179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.055002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.055293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.055738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.063558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.063975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.064789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.065013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.065144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.065319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.072421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.073859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:21.079823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:21.080818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 15:13:21.081840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:21.085826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 328/644] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:21.086717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:21.088073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:21.088945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:21.092078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:21.093831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 317/645] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:21.094717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.157230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.194241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9077082-eab3-4388-9312-48a827a9dc0c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.199808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf728c0-ef8c-4c83-b51c-e0450e431bc4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.222722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.231108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.231909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.232188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.232463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.237772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.238834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.239292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.239755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.244252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.245803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.253436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:21.262046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:21.263019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:21.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:21.264496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:21.270557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:21.272361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:21.272361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:21.272361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:21.272361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:21.273024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 318/646] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:21.273024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.273312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 329/647] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:21.273312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.369130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715b3b6-aff0-4467-89fa-4d5f77ce0c62 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.377953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4c86c-5028-471c-8921-e0baa30a75c6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.386715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.404170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.404843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.405120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.405390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.406237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:21.406537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.408094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.415124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.415767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.416049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.416391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.417110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:21.417409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.418870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.430155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 319/648] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:21.430155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.436326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 330/649] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:21.441371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.496944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.536707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb86f-277d-4281-8152-e579e3233483 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.561634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d97fc-249a-4d46-9a1f-d045cc95382b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.573440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.574116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.574394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.574691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.575437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:21.575734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.577110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.586087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 320/650] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:21.591640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.598007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.598415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.598722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.598996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.599724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:21.600031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.601419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.618097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 331/651] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:21.618097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.638569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.707582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2437ce1-a7e0-4b67-a472-06d882e58a84 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.722082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.725557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d5992-e626-42c4-a354-9700b9a43ceb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.748790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.749430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.749708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.749981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.764854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.765512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.765799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.766078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.766957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=063c589447364966b64fcc2da5c1aa02) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:21.767273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.768767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.782582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 332/652] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/role_assignments?user.id=063c589447364966b64fcc2da5c1aa02&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:21.782582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.837476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.839142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.895655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:21.912012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ea19-fa5c-43d1-9955-3602bb01953b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:21.954400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:21.955249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.955535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.955810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.968268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:21.969678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:21.977078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 321/653] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:21.977966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ac81bd2c-0383-4051-855e-440296599b6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:21.989861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:21.991182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:21.992186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:21.992186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:21.992186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:21.992186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:21.994133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 333/654] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:21.995655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799db54-d7c9-43f4-b36b-a6b051cf3d5f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:21.995957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799db54-d7c9-43f4-b36b-a6b051cf3d5f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:21.996228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799db54-d7c9-43f4-b36b-a6b051cf3d5f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:21.997638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 334/655] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:21.998192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799db54-d7c9-43f4-b36b-a6b051cf3d5f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.022999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.104216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ae36a-24e0-46ec-b77f-c981265cf841 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.137055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799db54-d7c9-43f4-b36b-a6b051cf3d5f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.146318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.147170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.147788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.148085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.159319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.161028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.163420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.174093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.174938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.175365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.175630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:22.177605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:22.178512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:22.179712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:22.183605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 322/656] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:22.184238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.187247 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.188733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6ef1de0c-9e18-4f5a-ab54-6904a7531388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:22.208270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:22.209383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:22.210461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:22.210461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:22.210461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:22.210461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:22.213852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 335/657] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:22.215579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.215866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.216149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.235107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:22.236219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:22.290195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a7c0a-4863-4053-8e1a-068d0777eb29 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.305967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.322762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 336/658] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:22.322762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.325153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.325828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.326110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.326380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.327144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:22.327451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.328939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.344930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 323/659] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:22.345896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.405312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.457582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb1f58-5d58-4d9e-b497-73077b66a5f9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.462579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca0fc9-0987-4fb7-b0e7-e995653b89f8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.494708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.495352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.495595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.495878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.496606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:22.496916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.498257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.498382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.498972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.499256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.499528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.500391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:22.500707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.502128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.511094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 324/660] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:22.511753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.529917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 337/661] 10.222.0.40 () {56 vars in 1137 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:22.529917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.588740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.623261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73b7a-3b5f-48cb-af76-4b734416333e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.633742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687514c4-0b73-477e-a577-49018c3b089c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.641670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.660742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.661523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.661801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.662077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.668893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.669702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.669979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.670251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.673279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:22.677395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.678873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:22.682042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:22.683099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:22.683961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:22.685275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 325/662] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:22.685275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:22.698290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:22.699335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 338/663] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:22.705636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.797921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c617fc-4bfa-4ac1-9996-f34eb88604a1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.802151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413010bb-9a4d-422e-a78d-2b5cda411312 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:22.819513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.833905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.835308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.835605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.835860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.838952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:22.839610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:22.839896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:22.840173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:22.842970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.845211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:22.862802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:22.863853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 326/664] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:22.870190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:22.907880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:22.909418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:22.918383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:22.991183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51784dcf-5933-4c73-a050-10a044fa770c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.027174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 339/665] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:23.028271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.035244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.035924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.036209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.036626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.037381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:23.037693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.039112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.046321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.053362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 327/666] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:23.054215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.135526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019a322-fa95-4133-b4f2-faca9f6e7656 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.151329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.154791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed00d6-697d-4513-a57d-813057b77ecc None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.174749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.175684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.175965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.176242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.177084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:23.177393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.178988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.192049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.192826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.193204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.194014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.194941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:23.195256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.196690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.197634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 340/667] 10.222.0.40 () {56 vars in 1139 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:23.198750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.219112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 328/668] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:23.219112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.286346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.312959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927ef46-2796-4736-a2f7-6da0be0574ae None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.323256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ac8b3-0bf2-4c15-a7dc-4a178c7bfccf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.352179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.353105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.353388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.353776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.361078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.361881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.362164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.362459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.365271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.367215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.373676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.375163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:23.385992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:23.387111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:23.388019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:23.388019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:23.388019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:23.388019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:23.390838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 341/669] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:23.391880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:23.392538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:23.393585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:23.394723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:23.394723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:23.394723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:23.394723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:23.398343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 329/670] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:23.399251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.414052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.498135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d858669-d633-42bd-b19c-cc6ad1d658c4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.501575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7bcf6d-729a-484c-bc2f-8165565936ce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.529965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.534891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.535561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.535948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.536222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.537069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=9fe7e2ad26c8483e8aecb89225f3b26e) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:23.537368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.538956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.539073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.539607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.539886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.540154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.550734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 330/671] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/role_assignments?user.id=9fe7e2ad26c8483e8aecb89225f3b26e&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:23.550734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.635508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.637050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.654297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.667126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158fe4d-43e5-443a-9037-3337665b0ac3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.703797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.704523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.704798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.705218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.705962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:23.706452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.707915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.726695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 331/672] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:23.727920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.795627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 342/673] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:23.801588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.835643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.864846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b5a32-5aa1-4751-b193-67ff8aaf74ab None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.904957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.905541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.905816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.906086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.906838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:23.907137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.908513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.911585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:23.921564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 332/674] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:23.922331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:23.923719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a26afe-06ac-469c-afad-b1575ba5626b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:23.968618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:23.969424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.969700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.969968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:23.976946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:23.978406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5bb47eb9-e893-4ec6-8c37-5b41d5b6c749 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:23.992955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:23.994042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:23.994993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 343/675] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:23.995473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078037-c00d-4eec-84a9-6e0411d7ed39 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:23.996057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078037-c00d-4eec-84a9-6e0411d7ed39 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:23.996057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078037-c00d-4eec-84a9-6e0411d7ed39 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.001407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 344/676] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:24.001407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3078037-c00d-4eec-84a9-6e0411d7ed39 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.033434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc15bad-8443-404f-b614-1d4737e2a4a2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.070428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.071368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.071647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.071920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.082963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.084429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.092848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:24.103413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:24.104495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:24.105399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:24.105399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:24.105399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:24.105399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:24.105399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 333/677] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:24.106309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.117127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3078037-c00d-4eec-84a9-6e0411d7ed39 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.154197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.155107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.155327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.155701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.157523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.167113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.168590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8e20e921-8d9a-485e-bc25-b6029e85748d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:24.185984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:24.187638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:24.188842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:24.191146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 345/678] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:24.192591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.193064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.193337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.209858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:24.211093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:24.217015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871ad52-1994-44d0-874e-8ac76067ccaa None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.259483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.260551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.260923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.261431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.262627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:24.263035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.264607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.277894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 346/679] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:24.278761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.290559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 334/680] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:24.291346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.358988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.384271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53bbf7-1c47-439b-99a4-f9396be1e1e4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.413965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24111517-b94d-4c03-b15c-a7c9a533b20c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.418939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.419733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.420113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.420485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.421314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:24.421725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.423251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.424832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.440797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 347/681] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:24.440797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.454218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.455097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.455476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.455751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.457338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:24.457885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.460544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.480846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 335/682] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:24.481771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.558166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451cbef-cd4b-4ae2-baaf-815ca520b45b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.593736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070673b9-78db-46e1-953a-1e20721a3b5c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.595938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.596744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.597138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.597629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.598507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=4cfd0067b59048f1850e8eb6e94640ef) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:24.598938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.600483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.603822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.614557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 348/683] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/role_assignments?user.id=4cfd0067b59048f1850e8eb6e94640ef&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:24.614557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.657121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.658586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.658871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.659141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.676242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:24.686152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:24.687387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:24.688348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 336/684] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:24.689059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.733561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.748664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b63e6-c29f-4422-b1f2-e844cf775419 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.789986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.791067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.791451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.791818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.800215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.801901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:24.824121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:24.825313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:24.826436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:24.827042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347234bc-16a5-4973-b175-7c7b463e4012 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.828970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 349/685] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:24.829384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:24.875665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.876324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.876602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.876871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.907704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:24.955715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ba09-3d2c-4a25-a475-c2f35b78000b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:24.976210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.978450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:24.992660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:24.993343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:24.993620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:24.993888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:24.994851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:24.995043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:24.996460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.005824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 350/686] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:25.006745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.014600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.114356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 337/687] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:25.116964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.119315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544a996-ecfa-4e94-a98b-f4e322585927 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.145605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.166313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.167375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.167833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.168279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.169818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:25.170334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.172856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.197241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 351/688] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:25.197241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.238964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d0bcb-eab5-49b3-9281-c1826a1d4a87 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.272671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.285188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.286224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.286680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.287123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.301843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.303420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.320410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977c42d-2190-4eac-b01d-da7f497248cb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:25.322375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:25.323484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:25.324590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:25.324590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:25.324590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:25.324590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:25.326206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 338/689] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:25.327127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.369910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.371543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.372004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.372421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.386766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.388784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:25.411222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:25.412391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:25.413520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:25.413520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:25.413520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:25.413520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:25.415118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 352/690] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:25.416098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.446582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a8627-4471-41d8-95bd-739e768dbe36 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.453584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.486581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.487955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.488385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.488802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.489780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=0d9d155ea97f422490871907f4018532) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:25.490268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.492321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.502792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 339/691] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/role_assignments?user.id=0d9d155ea97f422490871907f4018532&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:25.504427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.530117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83429e-bf8a-4ce4-8d45-88095aaa3d18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.569162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.569955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.570305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.570695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.571525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:25.571873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.573400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.587401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 353/692] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:25.599029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.600777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.655476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6147-fa81-4c2b-93a0-3f4d42be1fab None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.694675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.695987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.696470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.696943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.697903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:25.698412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.703436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.730258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 340/693] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:25.736441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.777106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.786549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad5ad7-2d5e-4811-bb22-31ce1114e58e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.836962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.836962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.836962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.836962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.848237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.848535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6673fce5-abd7-4e21-be13-3fcb64ae0921 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:25.863700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:25.864715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:25.866049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:25.869732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 354/694] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:25.871432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5062b6-c736-4f85-a566-bf5280cba754 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.871432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5062b6-c736-4f85-a566-bf5280cba754 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.871698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5062b6-c736-4f85-a566-bf5280cba754 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.877107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 355/695] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:25.877107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5062b6-c736-4f85-a566-bf5280cba754 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:25.902330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1a6cd-e3de-4b60-86ba-f6efbb12afa0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:25.904787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:25.948207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:25.949312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.949840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.950400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.959128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:25.961072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-6cdc3de8-7ad5-432d-9fc5-fab3f0ef359a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:25.977039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:25.978036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:25.978968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:25.981680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 341/696] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:25.983129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:25.983622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:25.984083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:25.993316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5062b6-c736-4f85-a566-bf5280cba754 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.004072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:26.005705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:26.028942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.029626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.029905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.030183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.031074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:26.031386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.033059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.056776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 356/697] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:13:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:26.057678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.073375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.082990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 342/698] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:26.091810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.163497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6b9c23-308a-491d-af3d-a65ced7b6723 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.201385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.202144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.202337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.202645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.203436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:26.203743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.205313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.208758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.218979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9a06c-ceb0-44a0-a4c0-f0e8c06c66da None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.225005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 357/699] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:26.225924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.257024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.257637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.257921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.258207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.304492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.333002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd1b6a-eb42-4635-a48b-722246cbac3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.341997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.343886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.368270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673867a-5914-4370-8141-f857232e7215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.369255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673867a-5914-4370-8141-f857232e7215 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.369598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673867a-5914-4370-8141-f857232e7215 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.369962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673867a-5914-4370-8141-f857232e7215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.382303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673867a-5914-4370-8141-f857232e7215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.383979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673867a-5914-4370-8141-f857232e7215 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6673867a-5914-4370-8141-f857232e7215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:26.402072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:26.403233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:26.404334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:26.404334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:26.404334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:26.404334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:26.410805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 358/700] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:26.411821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673867a-5914-4370-8141-f857232e7215 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.444368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.456663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 343/701] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:26.457851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.515190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673867a-5914-4370-8141-f857232e7215 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.539047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.550586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.551546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.552206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.552682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.553615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=19b3f6ab1b924bb5907bbb33db1a5b4e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:26.554038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.555639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.562065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 359/702] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/role_assignments?user.id=19b3f6ab1b924bb5907bbb33db1a5b4e&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:26.562988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.570458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f4c7-d50b-42cb-a307-0a2e9e25275b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.611396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.612465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.612940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.613509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.614550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:26.615031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.617030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.631152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 344/703] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:26.637500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.726212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36464f-5682-4f94-ae10-fafbce2db888 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.751835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.777837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.779042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.779325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.779615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.792056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.793530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.806375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490b009-57d5-48a1-988d-706f2f6d2c03 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-076a9238-277a-49f0-9fcf-21f6995ee8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:26.810926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:26.811821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:26.812743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:26.812743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:26.812743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:26.812743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:26.812743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 360/704] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:26.813418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6600-bab0-425e-bec8-3122f790938f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.813650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6600-bab0-425e-bec8-3122f790938f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.813917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb6600-bab0-425e-bec8-3122f790938f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.815543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 361/705] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:26.818974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6600-bab0-425e-bec8-3122f790938f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:26.854781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010e1c5-8fbc-4a5f-9c51-fa5bfde5d7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:26.855586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010e1c5-8fbc-4a5f-9c51-fa5bfde5d7e1 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.856203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010e1c5-8fbc-4a5f-9c51-fa5bfde5d7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.856203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010e1c5-8fbc-4a5f-9c51-fa5bfde5d7e1 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.856992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010e1c5-8fbc-4a5f-9c51-fa5bfde5d7e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:26.858492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010e1c5-8fbc-4a5f-9c51-fa5bfde5d7e1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:26.875946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:26.883288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 345/706] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:13:26 2022] POST /identity/v3/users => generated 254 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:13:26.884630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:26.884935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:26.885198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:26.915806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:26.917061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:26.980201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb6600-bab0-425e-bec8-3122f790938f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.052022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.052617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.052896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.053168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.053905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:27.054206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.055892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.057582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 346/707] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:27.069173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 362/708] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:27.074696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:27.084373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:27.137844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:27.220335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afc3f4-eb9c-4ed2-8052-0d0288118d9b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.232100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:27.236857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321d7bd-bc37-4573-a8f2-97c125594533 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.269006 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.269971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.270351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.271703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.279875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.281114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.282144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.282431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.282770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.283360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.284189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.286503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:27.299787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:27.300725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:27.301919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:27.302490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:27.303653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:27.304523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:27.305354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 347/709] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:27.305429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 363/710] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Jul 15 15:13:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:27.308810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:27.309851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:27.432031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747e6c1-9c54-474c-8eeb-f74a50131c00 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.437531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:27.444024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a007a-f21e-4b21-a234-70b083573a24 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.482665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.483501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.484352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.485071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.486367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:27.487317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.489387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.492536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.493339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.493720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.494097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.495932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:27.497742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.499313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.501597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 364/711] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Jul 15 15:13:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:27.507531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:27.529021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 348/712] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:27.535530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:27.567410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:27.653394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8409421-0b51-4838-bb72-2709c710406d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.701353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.702183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.702582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.703064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.703882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:27.704305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.705986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe9079-bbe5-4563-ad58-904402bdeb47 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.713428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 365/713] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Jul 15 15:13:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:27.733587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183bdb5f-b98b-4dd8-9f4b-c80c9eef9b33 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:27.794211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:27.795094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.795475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.795848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.796809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=063c589447364966b64fcc2da5c1aa02) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:27.797249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:27.798870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedcc6b-eda9-402e-90c6-92b2f49d7394 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:27.806307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 349/714] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:27 2022] GET /identity/v3/role_assignments?user.id=063c589447364966b64fcc2da5c1aa02&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:27.925172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c4244-c7fb-4e15-a08e-1122d7c33c6e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.925373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c4244-c7fb-4e15-a08e-1122d7c33c6e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.925761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c4244-c7fb-4e15-a08e-1122d7c33c6e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.927100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 366/715] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:27.933050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:27.933346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:27.933605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:27.960380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:27.961629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:27.983084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:28.036941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 350/716] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:28.058893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c4244-c7fb-4e15-a08e-1122d7c33c6e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:28.189396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:28.204452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c4244-c7fb-4e15-a08e-1122d7c33c6e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:28.254147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:28.254983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:28.255282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:28.255534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] PATH_INFO: `/v3/services/network` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:28.256203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:28.257590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:28.274834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:28.275885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:28.276746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 367/717] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:28 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:28.283651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:28.427015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca75a0b-c23b-4184-9138-3c7486ba7711 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:28.475281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:28.475961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:28.476246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:28.476515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:28.477222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] RBAC: Adding query filter params (name=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:28.477524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:28.478916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97856730-d6d3-42c8-833f-0bdbe4c3e552 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:28.486639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 351/718] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:28.492375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:28.603931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24d6c2-18ae-47ce-86a6-049cc9209022 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:28.646440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:28.647145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:28.647426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:28.647702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:28.648422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] RBAC: Adding query filter params (type=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:28.648728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:28.650121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6232c6-74bd-469b-8da5-b035323d9143 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:28.655455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 368/719] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:29.434433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55b2d-5ed6-4c16-ab54-3a1dc45ec50b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:29.435004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55b2d-5ed6-4c16-ab54-3a1dc45ec50b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:29.435004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b55b2d-5ed6-4c16-ab54-3a1dc45ec50b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:29.436386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 352/720] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:29.442038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f536b0-8738-4a97-90d0-0c34e7d32422 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:29.442479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f536b0-8738-4a97-90d0-0c34e7d32422 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:29.442686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f536b0-8738-4a97-90d0-0c34e7d32422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:29.463604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e0f536b0-8738-4a97-90d0-0c34e7d32422 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:29.465367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e0f536b0-8738-4a97-90d0-0c34e7d32422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:29.485608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:29.561226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 369/721] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:29.600663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55b2d-5ed6-4c16-ab54-3a1dc45ec50b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:29.627292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:29.716103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add26f24-2f0b-4d0c-9b36-b4f10e29008a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:29.716451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add26f24-2f0b-4d0c-9b36-b4f10e29008a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:29.716701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add26f24-2f0b-4d0c-9b36-b4f10e29008a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:29.718283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 370/722] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:29.727783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:29.728067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:29.728341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:29.740416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b55b2d-5ed6-4c16-ab54-3a1dc45ec50b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:29.759680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:29.761289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:29.794285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:29.795469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:29.795955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:29.796455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:29.816874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:29.825097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:29.826050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:29.826930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:29.829707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 353/723] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:29.838720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:29.899036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 371/724] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:29.940026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:29.975054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.017561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143cb12-ba21-4740-ab90-11689b409763 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.077601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.078279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.078582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.078859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.078981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.079616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:30.079926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.081565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.098841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 354/725] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:30.104127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.112992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67838db8-c7cd-45c1-ab8e-30d83366bef1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.156338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9dcfb5-a3da-4abc-96e3-d7947406032c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.157313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcfb5-a3da-4abc-96e3-d7947406032c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.157599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcfb5-a3da-4abc-96e3-d7947406032c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.158231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9dcfb5-a3da-4abc-96e3-d7947406032c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.170611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9dcfb5-a3da-4abc-96e3-d7947406032c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8a9dcfb5-a3da-4abc-96e3-d7947406032c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:30.178978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:30.180031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:30.180897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 372/726] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:30.181586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59cd3e-16bf-437e-8954-6912a9eac705 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.181663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59cd3e-16bf-437e-8954-6912a9eac705 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.182093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d59cd3e-16bf-437e-8954-6912a9eac705 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.183462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 373/727] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:30.184402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640a999-155d-44c9-b785-961feafe6f54 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.184690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640a999-155d-44c9-b785-961feafe6f54 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.184981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640a999-155d-44c9-b785-961feafe6f54 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.186418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 374/728] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:30.187141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640a999-155d-44c9-b785-961feafe6f54 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.220475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fd049-8fe3-4d65-b940-2da89e097443 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.256995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.258052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.258486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.258935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.266383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.272875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.274323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2b83594a-88fc-4689-af2a-07ec47555793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:30.292164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:30.293054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:30.294507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:30.294507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:30.294507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:30.294507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:30.294507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 355/729] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:30.294981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.295253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.295528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.299723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640a999-155d-44c9-b785-961feafe6f54 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.313991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:30.315730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:30.337329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.337997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.338279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.338574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.339331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:30.339634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.341108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18477827-a48d-48d1-b8a5-f2d25b212cf3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.354481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 375/730] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:30.354481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.354481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.354481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.359223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.374380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:30.375630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:30.397169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 356/731] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:30.398038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.462563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 376/732] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:30.462563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.521210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c1ed4-7c1e-4291-b06a-e89b97d94800 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.542703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.558807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.559482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.559761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.560024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.560752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:30.561059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.562577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.571297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c472a-b261-4e3a-b6c5-c545560fc6cf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.574174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 357/733] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:30.574983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.605000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.611119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.612270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.612968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.613147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.624157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.625501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:30.643565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:30.644778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:30.645718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:30.645718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:30.645718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:30.645718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:30.645718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 377/734] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:30.645718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.691546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba683-37ac-477d-b24c-d24c30021eb7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.726907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.727580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.727862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.728136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.728853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.730225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.758549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 358/735] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:13:30 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:30.758549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.760870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bf2bf-1889-445f-90ac-ffda46e8bdda None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.811377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.812508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.813457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.813739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.814011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.831265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:30.839947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:30.840977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:30.841871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 378/736] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:30.842606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.894959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e37a6-38c6-4868-891f-89363a895df3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:30.904296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:30.951660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:30.952720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:30.953028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:30.953263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:30.966806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:30.968292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:30.987235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:30.988256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 359/737] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:30.990438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:30.991515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b189311-6671-4e58-8585-83c74c8959d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.044003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.045692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.046157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.046638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.047816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:31.048745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.050677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.069376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 379/738] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:31.074505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.107638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.110562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a65a95-a8ad-4a26-b78b-3180d4ef13c2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.148202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.148997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.149346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.149691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.150531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:31.150932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.152465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.162416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 360/739] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:31.167575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.213057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.218131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079306b-1d6c-48ab-956e-29037b8753c1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.263860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.264898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.265603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.266399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.267541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:31.268080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.269779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.286137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe2568-300e-4deb-8465-3e6dd8e4fcd4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.290395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 380/740] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:31.291253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.329660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.330440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.330941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.331210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.345139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.347486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7ad52a33-0c0b-4618-be1c-6442db12d738 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:31.364040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:31.366145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 361/741] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5da1ff-b555-434d-af6d-0fd5f1ae38bc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.367496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5da1ff-b555-434d-af6d-0fd5f1ae38bc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.368195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5da1ff-b555-434d-af6d-0fd5f1ae38bc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.369169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 362/742] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:31.369881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5da1ff-b555-434d-af6d-0fd5f1ae38bc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.400309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.413356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef1119-c417-45fa-adf1-8a2214fd05c1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.458079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.458945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.459200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.459485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.470721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.472278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.485186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5da1ff-b555-434d-af6d-0fd5f1ae38bc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:31.490739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:31.491646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:31.497816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:31.497816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:31.497816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:31.497816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:31.497816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 381/743] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:31.497816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.507994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.527274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.528129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.528266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.528472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.598839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abbba3-ba90-4510-8962-c48d1f70fe03 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.600447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.601926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.625356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.633715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.634382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.634697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.634975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.635823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:31.636157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.637690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64261020-1b97-4997-8040-81ed365b24a0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.662651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 382/744] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:31.662651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.662651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.662651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.678284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:31.679541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:31.712269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 363/745] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:31.713201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.747478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.754562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 383/746] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 15:13:31.755190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.826541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154cd663-dff7-444d-9ccf-6a12a7b57bbd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.852404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.864188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b929c10-eef1-4cb7-8fd0-3bafdc9f0040 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:31.865434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.866091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.866385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.866687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.867395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:31.867697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.869074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:31.880972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 364/747] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:31.882072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:31.906551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:31.907207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:31.907486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:31.907754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:31.977939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:31.981667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:31.983452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.003497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09083f0a-0e46-4638-b50e-f0c32f98c968 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.039993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.040920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.041203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.041475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.052043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.054145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:32.071663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:32.072755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:32.073671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:32.073671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:32.073671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:32.073671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:32.073671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 365/748] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:32.073671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.101801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 384/749] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 15:13:32.101801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.117811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.187348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adb3c7-603d-4b69-b659-ff4b45f8ff37 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.200271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.204237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae676f-5514-4b68-978c-5991f053f1e1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.225215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.225892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.226173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.226447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.227202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.228820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.239542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.240711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.240992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.241262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.254868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 366/750] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:13:32 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:32.254868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.261266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.263096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:32.280391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:32.281440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:32.282377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 385/751] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:32.283007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.373969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.393724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0944-3485-4286-8fc9-5b39ef6d273f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.401650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e9ed3-fa33-43db-8279-5f948d60c502 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.446438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.446958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.447222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.447356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.447483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.447613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.447734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.447846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.448535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:32.448789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.450188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.463934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.465417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.466987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 367/752] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:32.468566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:32.487568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:32.489119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:32.490581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:32.490581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:32.490581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:32.490581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:32.490581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 386/753] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:32.490581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.506112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.581179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87689c3b-4488-4194-97d0-c94f24398047 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.619707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae9246-cc8d-4300-8a33-5a7b4fe4fa48 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.628917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.629534 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.629818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.630092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.630985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:32.631294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.632758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.653542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 368/754] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:32.654438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.663030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.663848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.664127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.664403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.665600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.672996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.674659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:32.689992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:32.691173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:32.692271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:32.696240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 387/755] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:32.697073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.776656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.790557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd113fe7-80d1-4a25-9b22-66014d0af0c7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.821158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061435b-9bf5-4c97-9ccd-fc8acaa25c13 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:32.825690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.826383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.826691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.826967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.827700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:32.828005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.829460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.842230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 369/756] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:32.843302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.872444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:32.873256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:32.873557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:32.873823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:32.889552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:32.891014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:32.915879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:32.916856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:32.917763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:32.917763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:32.917763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:32.917763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:32.918614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 388/757] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:32.919631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:32.923265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:32.966142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3648f7-e6ea-432f-a614-37cd5916362e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.001851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.002629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.002931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.003210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.003969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.004275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.005722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09d9fe-806e-41ad-b9a8-9b5d237de3c2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.019141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 370/758] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.025090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ec338-b767-44ec-bd29-26c4d82a8566 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.025381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ec338-b767-44ec-bd29-26c4d82a8566 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.025655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12ec338-b767-44ec-bd29-26c4d82a8566 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.026464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6086b2-52eb-4838-b140-4337a4980de8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.027102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 371/759] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:33.027631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ec338-b767-44ec-bd29-26c4d82a8566 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.055842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:33.068808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.069615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.069895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.070166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.077630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.079283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:33.095094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:33.096268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:33.097255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 389/760] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:33.097736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.137057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12ec338-b767-44ec-bd29-26c4d82a8566 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.164900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:33.174203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.174942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.175228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.175503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.176233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.176546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.177923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daaf732-130c-4b63-91c9-2e6c03639f52 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.190130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 372/761] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.192485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.193062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.193521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.204474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a3b83-56bd-4df4-a525-3aa0c602a897 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.210931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:33.212169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:33.240803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.241556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.241735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.242003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.242973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=19b3f6ab1b924bb5907bbb33db1a5b4e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.243280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.244757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.251603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 390/762] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/role_assignments?user.id=19b3f6ab1b924bb5907bbb33db1a5b4e&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.252191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.297255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 373/763] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:33.298717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.327827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:33.403085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30436f01-7d19-4f09-b364-98d2a2854fef None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.438462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb6187-21e2-44aa-9dee-766f028b59e2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.460612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.461658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.462048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.462448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.463221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:33.464482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.465038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.466804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.487612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 391/764] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.487612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.489584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.491003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.491288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.491562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.499386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.500856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:33.516541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:33.517490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:33.518931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:33.522662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 374/765] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:33.523786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.601167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd6334-c68f-44ba-9c8c-adcf3448ec2d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.609663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:33.633057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad4fb8-bdbd-44c9-bb55-4b3a2bca9e9d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.639983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.640946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.641347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.642117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.642619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.644683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:33.657792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:33.659305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:33.660358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:33.662462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 392/766] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:33.663692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.674997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.675940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.676493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.676945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.678075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=791ca74ca97b4667a5c170f443982daf) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.678594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.680457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.687320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 375/767] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/role_assignments?user.id=791ca74ca97b4667a5c170f443982daf&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.688353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.791160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:33.852966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65a6a9-b96d-49b3-b6ab-16203c98ffb1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.894945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df1cda-8124-4dee-bc6f-bb9e6c86b0a6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:33.896686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.897441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.897797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.898144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.899053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.899432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.900966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.915385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 393/768] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.923536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.943368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:33.944070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:33.944356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:33.944647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:33.945758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:33.945758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:33.954679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:33.954679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 376/769] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:33.960490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:33.972561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:34.058733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64878c-bf3f-41d6-98c7-5b669e5ec923 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:34.131410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:34.132780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.133263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.133732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/063c589447364966b64fcc2da5c1aa02/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.139523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=063c589447364966b64fcc2da5c1aa02, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:34.141408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:34.183536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 394/770] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:33 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/063c589447364966b64fcc2da5c1aa02/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:34.246111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb8932-5c0a-47c1-9c3d-f94297b5274e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:34.254470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:34.287457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:34.288151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.288429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.288701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.289433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:34.289736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:34.291193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:34.302863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 377/771] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Jul 15 15:13:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:34.304869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:34.409124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:34.424009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07134a20-8bcd-4069-8c76-49f9f86e7598 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:34.461532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:34.462200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.462478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.462856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.463648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] RBAC: Adding query filter params (interface=public, service_id=a6ef7d8a45a0447dbea264a67601e7e8, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:34.463956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:34.465343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c057e15f-0c8e-4b82-b9ff-8612f30222d1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:34.474917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 395/772] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:13:34 2022] GET /identity/v3/endpoints?service_id=a6ef7d8a45a0447dbea264a67601e7e8&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:34.480435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:34.584847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4050-c7df-4de8-b34c-12f35cc13cff None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.585418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4050-c7df-4de8-b34c-12f35cc13cff None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.585418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4050-c7df-4de8-b34c-12f35cc13cff None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.587962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 396/773] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:34.594150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.594427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.594726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.595316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b705a1-0f37-456c-acf0-a85894e4729e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:34.623548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:34.625102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:34.635758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:34.636434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.636756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.637041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.642561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:34.642561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:34.646056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 378/774] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:13:34 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:34.752350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 397/775] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:34.787502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:34.849137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:34.920237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16893d-e50e-4f90-a581-8f0c431547cd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:34.955610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:34.956548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:34.956828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:34.957098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] PATH_INFO: `/v3/services/network` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:34.957772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:34.959236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:34.974326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:34.975442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:34.976276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 379/776] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:34 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:34.981750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:35.086224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:35.108231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9c8e8-2a2d-49a4-9143-a6632635fe89 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:35.154719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:35.155400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.155682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.156011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.156782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:35.157094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:35.158769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:35.164115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 398/777] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:35.169748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:35.297695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:35.303101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8df1a6-a9e8-4adc-a432-62b2ab70bd26 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:35.347524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:35.348117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.348400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.348673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.349388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:35.349690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:35.351095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:35.358631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 380/778] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:35.365013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:35.474840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6d108-c463-47d1-8815-1c946e47ed63 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:35.517741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:35.518417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.518749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.519007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.519822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] RBAC: Adding query filter params (interface=public, service_id=63f79dc4f0f34184875c4b47aaf4597b, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:35.520144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:35.521548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290c1ec-c67c-49cf-b63b-2796d5c628fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:35.527758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 399/779] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:13:35 2022] GET /identity/v3/endpoints?service_id=63f79dc4f0f34184875c4b47aaf4597b&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:35.533029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:35.666779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d8f81-7e26-456d-b5f6-75b76ffa08d3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:35.718091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1834235-667b-4ae1-b7f2-f31671409269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:35.718825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1834235-667b-4ae1-b7f2-f31671409269 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.719113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1834235-667b-4ae1-b7f2-f31671409269 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.719393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1834235-667b-4ae1-b7f2-f31671409269 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.720099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1834235-667b-4ae1-b7f2-f31671409269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:35.721705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1834235-667b-4ae1-b7f2-f31671409269 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:35.732137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 381/780] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:13:35 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:35.910363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24906171-41bb-4a0c-9946-014519374acb None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.911113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24906171-41bb-4a0c-9946-014519374acb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.911113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24906171-41bb-4a0c-9946-014519374acb None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.912524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 400/781] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:35.924780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.924780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.925213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.928450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9613d-93c1-454d-a99f-0c6723435be1 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.928737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9613d-93c1-454d-a99f-0c6723435be1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.929018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9613d-93c1-454d-a99f-0c6723435be1 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.930588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 382/782] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:35.952032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:35.953297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:35.962305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:35.964420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:35.964710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:35.964989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:35.987432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:35.989369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:36.017368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:36.073642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 401/783] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:36.091099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 383/784] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:36.107411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.128253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.244620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8297ba-dafa-4aa4-a132-a002c42d0035 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.305442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d19fa-55aa-4d26-a912-3b898f14cdce None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.318622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.319462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.319728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.320003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.339812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:36.348299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:36.349316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:36.350170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 384/785] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:36.356578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.381378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.382594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.383072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.384919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.395872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:36.405468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:36.416723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:36.417685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:36.418562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 402/786] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:36.425816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.429346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:36.547969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053ea0b-db49-4519-9c72-0b5231dbe601 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.590857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.591642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.591964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.592306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.593068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:36.593378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:36.594988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:36.606997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228bd51-d25c-4e5a-a69a-3aa2081138dc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.608447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 385/787] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:36.610029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.675855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.676549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.676893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.677168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.677983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:36.678344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:36.679780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:36.694276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 403/788] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:36.699852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.709442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:36.715433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b444949-cfae-4a42-bd6a-5d6c4d53517d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.751038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.751924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.752216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.752686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.763586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:36.765060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:36.784202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:36.785182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:36.786061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:36.786061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:36.786061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:36.786061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:36.790255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 386/789] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:36.791585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.829538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:36.870618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcedad-d126-45e4-8f9e-cd8fc1ba56b9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.899594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd10edf-9134-42f3-9e79-02fa4440ee1d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:36.906465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.907641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.908141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.908608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.926114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:36.927760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:36.938370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:36.949313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:36.950217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.950765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.951237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.952162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:36.952662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:36.954296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcda9e-4e78-4042-b5d0-db8ebccf61c9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-405ba716-4a26-41f6-82f0-c81ea51fa6cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:36.961530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:36.962475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:36.963412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:36.963412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:36.963412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:36.963412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:36.966764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 404/790] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:36.968284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d36f59-b968-4542-be4e-5799496a70c8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.968781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d36f59-b968-4542-be4e-5799496a70c8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.969308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d36f59-b968-4542-be4e-5799496a70c8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:36.971478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 405/791] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:36.972378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d36f59-b968-4542-be4e-5799496a70c8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:36.974114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 387/792] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:36.976538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:36.976826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:36.977098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.002864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:37.004805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:37.083985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 388/793] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:37.085186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.106704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d36f59-b968-4542-be4e-5799496a70c8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.129666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.150765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.151839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.152341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.152809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.153785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:37.154285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.156001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.175667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 406/794] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:37.176964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.224540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b3e06-131f-469f-beb7-1fec0bbe9d5d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.251222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.270963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.272079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.272923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.273341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.284130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.285576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.300361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049b972-7b7a-4028-bbc1-96c88feebea6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:37.300695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:37.301621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 389/795] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:37.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.341828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6c220-8586-4a00-8bde-b625e7c9a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.342575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6c220-8586-4a00-8bde-b625e7c9a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.342857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6c220-8586-4a00-8bde-b625e7c9a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.343122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6c220-8586-4a00-8bde-b625e7c9a2e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.354509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6c220-8586-4a00-8bde-b625e7c9a2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fcd6c220-8586-4a00-8bde-b625e7c9a2e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:37.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:37.363277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:37.364119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 407/796] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:37.365735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c69f9-057f-4c55-86d7-82dc2e486f53 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.365735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c69f9-057f-4c55-86d7-82dc2e486f53 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.365901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c69f9-057f-4c55-86d7-82dc2e486f53 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.371283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 408/797] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:37.371999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c69f9-057f-4c55-86d7-82dc2e486f53 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.411697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.415122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc2af-924f-4d7d-8a7f-e559fea7927b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.450432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.451283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.451565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.451835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.459620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.461416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:37.476281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:37.477444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:37.478720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:37.482199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 390/798] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:37.483060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.488652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c69f9-057f-4c55-86d7-82dc2e486f53 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.525860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.526716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.527071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.527416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.528425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:37.528643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.529092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.530103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43ac64-59ca-455f-92d2-22f4e3088729 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.551348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 409/799] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:37.557437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.558297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.558839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.585866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:37.588050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:37.614721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d8ebe-fc4e-4464-92ba-ef9553bc1834 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.674224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.674934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.675218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.675495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.676263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:37.676642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.678000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.682741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 410/800] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:37.682741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.696074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 391/801] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:37.697240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.706735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.785337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158158e-0fe5-4d0f-98c9-161370a466b0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.813850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f843c-dfa8-4073-bcfc-bdad6f6e622b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:37.817675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.823471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.824155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.824435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.824712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.825584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:37.825890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.827370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.847292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 411/802] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:37.848144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.859270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:37.860129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:37.860671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:37.860671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:37.872176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:37.873638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:37.891833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:37.893410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:37.894351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:37.894351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:37.894351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:37.894351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:37.894351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 392/803] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:37.894672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:37.946507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:37.966222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52866f-36af-473e-b6de-bd95e65db87b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.001325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.002273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.002594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.002906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.003087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b68fc9-0399-49fd-9936-d8d30ecf72f2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.014384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.015977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:38.033767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:38.034751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:38.035690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:38.035690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:38.035690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:38.035690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:38.035690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 412/804] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:38.035966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.045741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.046560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.046875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.047132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.047781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.049280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:38.062199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:38.063406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:38.064451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 393/805] 10.222.0.40 () {56 vars in 1129 bytes} [Fri Jul 15 15:13:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:38.065467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.086274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.149962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40263a0-2008-495e-bc5a-d0bcfda7b498 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.173996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3be826-498e-4a95-b9af-68a7ecacce87 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.184123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.184947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.185232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.185509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.187737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.197404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.198970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.209681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.210350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.210671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.210942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.211680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:38.211991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.213350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:38.216926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:38.218221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:38.219445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:38.219445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:38.219445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:38.219445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:38.219445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 413/806] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:38.219445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.229364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 394/807] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:38.230185 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.308151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.321447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1455225-0a79-4ba2-bbfc-1dff9fb68de8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.331190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7d048-7fbf-4a36-9a22-290a548c151a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.354753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.355422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.355711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.355975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.356701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:38.357011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.358440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbb451-f7c3-4f23-82c8-fc78c3ccd430 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.365974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.366675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.366959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.367232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.367933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:38.368240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.369602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.376001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 414/808] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:38.376001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3b76-8829-4a07-be3d-ace981d728d5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.376001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3b76-8829-4a07-be3d-ace981d728d5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.376001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc3b76-8829-4a07-be3d-ace981d728d5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.376001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 415/809] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:38.376001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3b76-8829-4a07-be3d-ace981d728d5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.379212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 395/810] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:38.380029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.430602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.481144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92092359-a796-42fe-9a28-e90ab89c2692 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.487148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc3b76-8829-4a07-be3d-ace981d728d5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.514653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.515453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.515740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.516017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.520486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.521145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.521424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.521699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.522413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:38.522743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.523158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.524147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.524581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.536075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 416/811] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:38.536856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0420c239-aa91-4b79-9456-fae38601d507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:38.538886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:38.539787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:38.540697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 396/812] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:38.541502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.541502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.541771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.546867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.559459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:38.560667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:38.626506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 397/813] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:38.627349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.641505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a42e-5de3-495f-9056-e1afb74f6658 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.735025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4361848-9fe1-4ee8-a66e-02766de61c83 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.768988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.769209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:38.775319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.776002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.776288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.776560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.777277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:38.777581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.778993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.784899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 398/814] 10.222.0.40 () {56 vars in 1124 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:38.785685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.803205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.803990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.804292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.804547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.811967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.813464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:38.828064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:38.829180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 417/815] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:38.830628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.891702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f948f-0a7b-4721-bfad-673c60af1ae3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.925585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.926412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.926724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.926997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.931694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88cdc6-c017-4131-9379-9a00f019762a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:38.933647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.935124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:38.949507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:38.950559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 399/816] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:38.951485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:38.974162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:38.974843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:38.975126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:38.975398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:38.976182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:38.976517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:38.977941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:38.990441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 418/817] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:38.991197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.008817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.021565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.053173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647792-70e1-4168-998b-d7d39abc8ea7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.087073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.087898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.088150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.088424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] PATH_INFO: `/v3/services/network` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.089077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.090448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.095275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92d079-ac17-4e8a-9a6e-22ce5e01f40d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:39.101989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:39.103045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:39.103966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 400/818] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:38 2022] GET /identity/v3/services/network => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:39.104768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.135316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.135993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.136280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.136568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.137317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.138782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.159491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 419/819] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:13:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:39.160314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.218700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57beb48-4715-4116-81c1-e86a57e36a07 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.242903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.267359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.280366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.281124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.281409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.281683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.282575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:39.283071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.284602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.292453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b1ffc-e4ef-4607-9d2a-b1c0fa20227e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.308724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 401/820] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:13:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:39.309926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.338957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.339894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.340188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.340448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.341235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:39.341543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.343024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.355889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 420/821] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:39.356548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.422511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debe212-f0b8-40da-81fb-9933510908cd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.458236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ccb9a-7670-4371-a86c-174c900aa05c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.459858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.460868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.461153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.461428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/19b3f6ab1b924bb5907bbb33db1a5b4e/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.463451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=19b3f6ab1b924bb5907bbb33db1a5b4e, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.464882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.490557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 402/822] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:39 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/19b3f6ab1b924bb5907bbb33db1a5b4e/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:39.490557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.507582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.508270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.508606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.508960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.509907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:39.510277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.511838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.517778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 421/823] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:39.518990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.529398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.546323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.631992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b26387-f99f-4f73-8b92-0bf4382afc49 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.639244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b0133-1bc9-41b0-88d0-19834bf64911 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.672361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.673043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.673316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.673699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.683640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.684564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.684896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.685169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/791ca74ca97b4667a5c170f443982daf/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.711742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=791ca74ca97b4667a5c170f443982daf, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.714256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.734874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 422/824] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:39 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/791ca74ca97b4667a5c170f443982daf/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:39.736130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.756019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.758512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.809654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.821632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:39.884325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7cf0e-6c9d-46ab-83ea-bd22f2c513f7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:39.932343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:39.933459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:39.933945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:39.934441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:39.935428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] RBAC: Adding query filter params (type=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:39.935938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:39.938037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:39.949598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 423/825] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:39.954706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:39.979003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 403/826] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:13:39.984644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:40.085610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39979b2e-62c9-4777-8c39-304da7112eed None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:40.108748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:40.125143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:40.125922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:40.126227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:40.126546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:40.127378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:40.128772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:40.129858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa76e9-4b62-4f22-832b-261ed0ae8db0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:40.149411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 424/827] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:13:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:13:40.175638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:40.183145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:40.184384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:40.184889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:40.185366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:40.198628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:40.200503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:40.222432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:40.223701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:40.224657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:40.224657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:40.224657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:40.224657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:40.227229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 404/828] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:40.234584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:40.366402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036bf59-48de-4366-a89b-4240874498f4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:40.402602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:40.403283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:40.403559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:40.403828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:40.404554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:40.404856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:40.406242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:40.419859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 425/829] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:40.425810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:40.558383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb7fd8-d322-4d4c-ab2e-02733643e324 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:40.594300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:40.595251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:40.595599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:40.596251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:40.603452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:40.605965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:40.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:40.627752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:40.628945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 405/830] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:40.632891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:40.673123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:40.750684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9588d-59c4-46e9-8caa-67249b3dd1c4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:40.785818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:40.786598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:40.786948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:40.787289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:40.788142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:40.788514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:40.790009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b112b-9a7c-4205-970f-a9d74e7f515d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:40.802383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 426/831] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:40.810761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:40.856522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:40.921354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a0dd8-84b6-4409-816f-2039d0842b9d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:40.956878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:40.957546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:40.957830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:40.958104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:40.958974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=063c589447364966b64fcc2da5c1aa02) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:40.959291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:40.960725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521028b0-9447-48ae-8a37-ca677aaa909c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:40.967224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 406/832] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:40 2022] GET /identity/v3/role_assignments?user.id=063c589447364966b64fcc2da5c1aa02&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:41.479087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b83b-f068-46bb-9450-129e487a049a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:41.479648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b83b-f068-46bb-9450-129e487a049a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:41.479648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291b83b-f068-46bb-9450-129e487a049a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:41.481428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 427/833] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:41.487186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:41.487625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:41.487836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:41.505510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:41.506774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:41.575813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 407/834] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:41.597206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291b83b-f068-46bb-9450-129e487a049a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:41.700516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:41.709198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291b83b-f068-46bb-9450-129e487a049a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:41.745328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:41.746061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:41.746345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:41.746652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:41.747314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:41.748709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f7ddabca-a41e-4b2d-9fc9-85d463671534 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:41.761035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:41.762057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:41.762908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 428/835] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:41.771898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:41.805849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05077f-a733-4023-89cb-8a5c7952485b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:41.806051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05077f-a733-4023-89cb-8a5c7952485b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:41.806325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05077f-a733-4023-89cb-8a5c7952485b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:41.807915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 429/836] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:41.810822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:41.813097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:41.813390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:41.813666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:41.832008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:41.833693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:41.879661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2f307-e019-43c6-a6b4-7e41367f90c6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:41.915345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:41.916052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:41.916334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:41.916603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:41.917315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:41.917619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:41.919031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:41.925128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 430/837] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:41.930674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 408/838] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:41.931843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:41.959173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.038228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.048763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c41e7-aaba-4152-b41e-a53b952f61e0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.052928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.055982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12a19f-9271-4c71-aace-97329c1c0f2f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.084139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.084812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.085098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.085373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.086086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:42.086391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.087805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672f370-b124-499d-8f69-82e00717b8c6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.090451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.091272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.091549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.091820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.092504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 431/839] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:42.093765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4f4e8-5df8-4826-b9d4-5cd546b928f8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.094050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4f4e8-5df8-4826-b9d4-5cd546b928f8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.094321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4f4e8-5df8-4826-b9d4-5cd546b928f8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.095804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 432/840] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:42.104368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.104745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.105094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.112621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:42.121288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:42.122505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:13:42.123724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:42.125678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:42.130581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:42.130771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 409/841] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:42.135830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.208123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 433/842] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:42.233426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.268445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae609-a173-4c9c-9ce5-b7b4cb5d1c71 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.312967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.313573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.314069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.314190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.314933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:42.315242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.316624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.329157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 410/843] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:42.338272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.338450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.354743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e0140-82f9-4539-913d-1b0d2714c3f9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.367814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.399658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.400843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.401129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.401400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.416206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:42.426173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:42.427177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:42.428016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 434/844] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:42.435715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.457748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1ef94-3a97-45f3-9da0-2c2f13c6b30d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.499502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.500309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.500692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.500971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.514093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.515771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:42.536919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:42.539701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:42.541756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:42.541756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:42.541756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:42.541756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:42.541756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 411/845] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:42.547759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.553781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4720b16-1f78-4ca5-b785-d71ba0a0701b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.596488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.597086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.597465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.598198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.599105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:42.599413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.601174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.613376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 435/846] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:42.618736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.643796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.646748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.684227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5536-a16f-4eba-94e4-891b766642c5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.733005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.734221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.734733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.735184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.735720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac92b-9314-4082-936b-d4d225f39dc1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.736353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:42.736839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.740035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.760955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 412/847] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:42.762940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.775976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.776830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.777105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.777406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.789376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.790948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:42.808797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:42.809802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:42.811144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:42.811144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:42.811144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:42.811144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:42.814733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 436/848] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:42.816026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:42.877898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.901155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed1f4-321a-464c-906e-811721e3c428 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.929495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3d2f-bc73-4c4d-9747-ab69c3a8cb58 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:42.945926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.946886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.947070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.947305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.950011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:42.959174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.961286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.975175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:42.976075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:42.976527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:42.976940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:42.977811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:42.979013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 15:13:42.980193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:42.980980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:42.981439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:42.986819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 413/849] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:42.987954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:42.991181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.009171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 437/850] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:43.011146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.101567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fc6f5-6388-4180-9fcc-53ec19708713 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.136909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.137500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.137781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.138053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.138915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:43.139221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.140173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8eb523-866d-489a-8787-c367a6492595 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.140675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8545ab-7448-4490-93c4-a870ea6421ce None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.163367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 414/851] 10.222.0.40 () {56 vars in 1135 bytes} [Fri Jul 15 15:13:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:43.163367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63285-dfbc-418c-bf48-042da916aef1 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.163367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63285-dfbc-418c-bf48-042da916aef1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.163367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc63285-dfbc-418c-bf48-042da916aef1 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.163367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 415/852] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:43.165748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.167179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63285-dfbc-418c-bf48-042da916aef1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.185336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.186398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.186709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.186982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.195191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.196969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d3fe9d0f-a825-40c0-aca7-77165bf22009 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:43.212292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:43.213296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:43.214251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 438/853] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:43.214917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c822579-b468-416d-af30-724c411e024e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.214917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c822579-b468-416d-af30-724c411e024e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.215894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c822579-b468-416d-af30-724c411e024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.230552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.231440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2c822579-b468-416d-af30-724c411e024e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:43.232991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2c822579-b468-416d-af30-724c411e024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:43.305758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc63285-dfbc-418c-bf48-042da916aef1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.319348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 439/854] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:43.333239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c822579-b468-416d-af30-724c411e024e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.351520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.352336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.352632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.352896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.428332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.429707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.443186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c822579-b468-416d-af30-724c411e024e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.454679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.481867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.482805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.483678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.483835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.484111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.485021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:43.485249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.486809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.504367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 440/855] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:43.505167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.541522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 416/856] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:13:43.543065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.606125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6468d2bc-940d-4691-b29d-f5b125b7e300 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.641967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.642905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.643176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.643441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.644093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.644320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c83bf0-ca29-4dc3-a1bd-21fce11670ea None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.645475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:43.658192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:43.659328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:43.660329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:43.664355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 441/857] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:43.665196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.684214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.685077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.685503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.685920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.696024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.709235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.758076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.759563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.767414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9575b0-7da8-405b-a58c-2df1f6ec9cc0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.804965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.805802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.806090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.806365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:43.818955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:43.820435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:43.839332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:43.840288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:43.841965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:43.841965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:43.841965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:43.841965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:43.841965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 442/858] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:43.841965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.869859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 417/859] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:13:43.871148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:43.924752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.947335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:43.964556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96795c9-2961-4c8e-97b3-55cf191de332 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.982243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a399cf7-76d0-40e4-b225-96c0c7de304b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:43.998733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:43.999404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:43.999685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:43.999983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.000693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.001005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.002457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.007025 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 443/860] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:44.007803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.026070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.026764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.027046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.027331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.028231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.028356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.029670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.041724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 418/861] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:44.046832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.116410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877727c-2550-4274-9f93-6c7a5ec2144f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.153664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.154478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.154801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.155079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.155980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c973c0-4e93-4181-b76d-95d7329b1033 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.168344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.169120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:44.169824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.176901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:44.189045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:44.190232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:44.192005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:44.192005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:44.192005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:44.192005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:44.192005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 444/862] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:44.192005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.199420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.200173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.200522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.200859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.201776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.202078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.203895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.209701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 419/863] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:44.210475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.322449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774cb83-f7ea-4d8d-86ae-ba19c8159734 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.324665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c0828-7d1d-45e6-8265-70aeac50b2b9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.360881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.361843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.362057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363c051-d245-424f-9232-1550936b2188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.362324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.364913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c051-d245-424f-9232-1550936b2188 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.365041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.365134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c051-d245-424f-9232-1550936b2188 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.365134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363c051-d245-424f-9232-1550936b2188 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.365134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363c051-d245-424f-9232-1550936b2188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.365134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363c051-d245-424f-9232-1550936b2188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.366046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363c051-d245-424f-9232-1550936b2188 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.373844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.375593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.384855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 420/864] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:44.384855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba45f-f84e-4e95-9dfc-f80ace4370c0 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.384855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba45f-f84e-4e95-9dfc-f80ace4370c0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.384855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba45f-f84e-4e95-9dfc-f80ace4370c0 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.384855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 421/865] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:44.387357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba45f-f84e-4e95-9dfc-f80ace4370c0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-242efb08-6232-4cc7-b28e-ae169f043cf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:44.391086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:44.392326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:44.393419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:44.393857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 445/866] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:44.394874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.395159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.395432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.413539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:44.414760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:44.429741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:44.474845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:44.494719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 446/867] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:44.494719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.502151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba45f-f84e-4e95-9dfc-f80ace4370c0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.539819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.540634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.540974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.541261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.548539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.550331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:44.564693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:44.565631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:44.566587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 422/868] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:44.567075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.619857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de715304-ae8a-49ec-9f92-05f3c51c2d3a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.662706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.663736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.664293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.664568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.665362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.665667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.667618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:44.669723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.686595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 447/869] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:44.687018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb3ae7-ef8f-4171-a1c2-c922ad001320 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.687234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.724301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.725134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.725428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.725795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.726595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.728056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.739270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:44.744188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 423/870] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:13:44 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:44.744841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.804912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661a00-b2fb-4f75-8080-8ea61d627a8e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.851186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.851861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.852134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.852405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.853199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.853558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.855060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.869401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 448/871] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:44.876415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:44.903930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e404ab1-09c1-462e-bf4a-c407a64614e1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:44.958784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:44.960042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:44.960042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:44.960238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:44.960988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=19b3f6ab1b924bb5907bbb33db1a5b4e) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:44.961309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:44.962806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f8f55-04d9-4b2f-8052-c8a4bc75e0cb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:44.970454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 424/872] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/role_assignments?user.id=19b3f6ab1b924bb5907bbb33db1a5b4e&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:45.000088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71345-b156-4b24-9cc6-d9348834199f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:45.010482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:45.044185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:45.045251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:45.045782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:45.046254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:45.047365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=791ca74ca97b4667a5c170f443982daf) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:45.047941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:45.049725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcd4d0-420d-48ce-a5df-278c109806c2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:45.057331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 449/873] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:44 2022] GET /identity/v3/role_assignments?user.id=791ca74ca97b4667a5c170f443982daf&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:46.227683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9a8ca-5b72-495d-88b0-1b249b9cafdc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:46.228217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9a8ca-5b72-495d-88b0-1b249b9cafdc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:46.228217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9a8ca-5b72-495d-88b0-1b249b9cafdc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:46.229867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 425/874] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:46.235715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0112b2-88da-4ca0-8aef-d134e0a07fa0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:46.236212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0112b2-88da-4ca0-8aef-d134e0a07fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:46.236291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0112b2-88da-4ca0-8aef-d134e0a07fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:46.255702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7a0112b2-88da-4ca0-8aef-d134e0a07fa0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:46.256910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a0112b2-88da-4ca0-8aef-d134e0a07fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:46.328280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 450/875] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:46 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:46.346733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9a8ca-5b72-495d-88b0-1b249b9cafdc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:46.369841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:46.453311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9a8ca-5b72-495d-88b0-1b249b9cafdc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:46.490151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6889a4f-6dff-4c37-a1a6-9a17798a3cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:46.490777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889a4f-6dff-4c37-a1a6-9a17798a3cc9 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:46.491061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889a4f-6dff-4c37-a1a6-9a17798a3cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:46.491335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889a4f-6dff-4c37-a1a6-9a17798a3cc9 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:46.492072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889a4f-6dff-4c37-a1a6-9a17798a3cc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:46.493570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889a4f-6dff-4c37-a1a6-9a17798a3cc9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:46.509267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 426/876] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:13:46 2022] POST /identity/v3/services => generated 269 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:46.895108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3f5f-319b-4474-a928-345e08adcee3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:46.895525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3f5f-319b-4474-a928-345e08adcee3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:46.895927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3f5f-319b-4474-a928-345e08adcee3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:46.897766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 451/877] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:46.903158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b294ae6-67b9-464c-9196-4c7bd9ce8af5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:46.903557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b294ae6-67b9-464c-9196-4c7bd9ce8af5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:46.903930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b294ae6-67b9-464c-9196-4c7bd9ce8af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:46.921325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4b294ae6-67b9-464c-9196-4c7bd9ce8af5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:46.922547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4b294ae6-67b9-464c-9196-4c7bd9ce8af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:46.998746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 427/878] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:47.021250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3f5f-319b-4474-a928-345e08adcee3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.131361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.132632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3f5f-319b-4474-a928-345e08adcee3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.168701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.169886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.170384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.170888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.171756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.173419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.176950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f303c-7670-44e6-aec9-9e9abc8192dd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.177244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f303c-7670-44e6-aec9-9e9abc8192dd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.177524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f303c-7670-44e6-aec9-9e9abc8192dd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.179242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 428/879] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:47.184309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c154af-7686-4b38-9cb6-efb7067faa6e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.184617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c154af-7686-4b38-9cb6-efb7067faa6e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.184902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c154af-7686-4b38-9cb6-efb7067faa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:47.189167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:47.190385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:47.191295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.194192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 452/880] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:47.198550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.200656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.207940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f2c154af-7686-4b38-9cb6-efb7067faa6e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:47.209341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f2c154af-7686-4b38-9cb6-efb7067faa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:47.286861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 429/881] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:47.288316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c36dd-d798-41a5-91f4-596dde8a216b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.288603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c36dd-d798-41a5-91f4-596dde8a216b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.288879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c36dd-d798-41a5-91f4-596dde8a216b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.290304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 430/882] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:47.297103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.297354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.297630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.316069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:47.317541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:47.320093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387a73c4-5931-462c-9400-0c83c670fec0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.357618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.358291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.361181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.361181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.361181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:47.361181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.362538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.372876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 453/883] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:47.372876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.396175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 431/884] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:47.398008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.407759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.472570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6dd13-f1c6-45d2-8f49-946aa1591376 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.479954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.508827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.509661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.509946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.510209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.510910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.512310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.519585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bfbb0-de59-4a2b-8add-d20fc7acf740 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:47.524719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:47.525714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.526590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 454/885] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:47.527420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.562803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.563508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.563824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.564177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.565026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:47.565308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.566720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.577069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 432/886] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:47.577069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.634852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.647019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b294bba-ad44-49ab-ba7c-4181b7e7bc1a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.686092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.686940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.687218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.687487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.691554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c01bff-76f6-4992-8a6b-6368fec3f13a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.698002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:47.706334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:47.707531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 455/887] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:47.708372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.727906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.734444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.735218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.735573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.736120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.736554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:47.736862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.738231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.744253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 433/888] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:47.747348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.824169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d9c7e-7c0f-4747-b257-2916ce0caa03 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.859246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefca687-171e-4cdb-92f4-c0da340c8616 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:47.860223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.861035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.861320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.862246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.862427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] RBAC: Adding query filter params (interface=public, service_id=a1cd6dc754a9485583f7fce6bd8196c1, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:47.862727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.864470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.870169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 456/889] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/endpoints?service_id=a1cd6dc754a9485583f7fce6bd8196c1&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:47.870810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.888009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.900776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:47.901450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:47.901733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:47.902010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:47.902788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:47.903094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:47.904470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:47.917591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 434/890] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:47.917591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:47.976745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:47.990992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6bb27-bbe8-4358-8f5b-10202d33e352 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.024213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedd7ce-2657-43bc-b38d-9bc4f6eaad72 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.026578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.027382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.027501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.027765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.028477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:48.028783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.030177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.042559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 457/891] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:48.042559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.076140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.076933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.077291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.077683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.078467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.080162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.087214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 435/892] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:13:47 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:13:48.148304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:48.177888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accab642-35c8-449b-8f61-174736a22c8d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.221170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.222124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.222412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.222720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.235558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.237071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:48.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:48.258215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:48.259158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:48.259158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:48.259158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:48.259158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:48.262950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 458/893] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:48.264785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.370072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd967be6-592c-4cb9-b170-4fa5d7d2deed None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.390732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:48.406770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.407495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.407776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.408050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.408792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:48.409111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.410499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f06e7a-53dd-420c-bc80-bc4099f310be None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.423691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 436/894] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:48.429358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.535858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6da82-a505-4df4-80c8-2c03ece744db None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.551132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a2d62-b042-4c93-984e-46128e840153 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.551466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a2d62-b042-4c93-984e-46128e840153 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.551692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a2d62-b042-4c93-984e-46128e840153 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.554409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 437/895] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:48.558471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.558793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.559072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.566415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:48.572983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.573993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.574439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.574989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.577706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:48.579093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:48.586547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.586547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:48.608055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:48.609077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:48.610071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 459/896] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:48.618403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.666211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 438/897] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:48.688131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.738900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34426285-8e5d-45a7-87f5-d939f93de90d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.786236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.787297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.787821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.788295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.789343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:48.789862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.791547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.795592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61183c00-6cd6-4e4f-a924-eb063b9c57d9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.807428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:48.816621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 460/898] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:48.818574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.839844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.840970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.841404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.841822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:48.842659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:48.844255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:48.857433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:48.858486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:48.859409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:48.867387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 439/899] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:48.868506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:48.879837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:48.956692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da521a-c0d4-4150-baed-caa44bdb3339 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.992226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1bce3-f427-4333-869b-af022f6cbbf9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:48.993046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:48.993899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:48.994365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:48.994848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.035998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.036735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.036965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.037240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.037961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:49.038271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.040127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.051295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 440/900] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:49.052635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.075582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.077054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.097808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:49.137066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:49.167866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5929e7-37a4-4b64-8629-99813295a97e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.197217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 461/901] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:48 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:13:49.198584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.207682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.208730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.209227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.209770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.210756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:49.211452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.213193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.224975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 441/902] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:49.226446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.304173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341e817-43b5-4a59-adf6-2d2df8a56249 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.338028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd65ae-274c-46c9-8aef-e84bb8c4b0b5 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.339080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.339940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.340221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.340537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.340914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:49.352568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.354160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.362960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:49.372811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:49.374074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:49.375406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:49.375406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:49.375406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:49.375406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:49.379601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 462/903] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:49.381082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.385036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.386302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.386820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.387313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.388362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] RBAC: Adding query filter params (interface=public, service_id=42b9a42a4ef04239bda6e8cbee2271d9, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:49.388924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.390673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.404061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 442/904] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/endpoints?service_id=42b9a42a4ef04239bda6e8cbee2271d9&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:49.405337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.483047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53ea4-1935-491a-826e-16d53a86ef7a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.515894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398193c7-7827-4276-92c9-a63d572294b8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.519665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.520076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.520360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.520635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.521365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:49.521671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.523119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.539876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 463/905] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:49.540888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.555287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.556364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.556851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.557349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.558236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.560049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.574683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 443/906] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:13:49.608724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:49.656638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcebe3e-b0e6-4f8e-b8da-bc56425e26f6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.694196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.695056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.695336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.695612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.703825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.705442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:49.721478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:49.722493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:49.723460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:49.726656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 464/907] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:49.729270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:49.763188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:49.841740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add79947-6df1-4819-8410-8561e9e13d29 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:49.878056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:49.879130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:49.879619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:49.880097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:49.881097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:49.881631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:49.883297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b05873-0d95-40ea-b92e-ec9d82ca6b56 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:49.896653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 444/908] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:49.904871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:50.015206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f26f3-180f-4da8-8456-4a486918df9a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:50.036789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:50.051713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:50.052375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.052653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.052928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.053771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] RBAC: Adding query filter params (role.id=5c1724e5bd744c02830f4aebfeec8458, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=791ca74ca97b4667a5c170f443982daf) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:50.054082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:50.055566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee429d6d-2f90-4407-a2f7-374ee72bab73 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:50.066667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 465/909] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:49 2022] GET /identity/v3/role_assignments?user.id=791ca74ca97b4667a5c170f443982daf&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=5c1724e5bd744c02830f4aebfeec8458 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:50.115251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c0133-0ec8-4062-8122-f3fc81b2ffbc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.116008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c0133-0ec8-4062-8122-f3fc81b2ffbc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.116505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c0133-0ec8-4062-8122-f3fc81b2ffbc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.118591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 445/910] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:50.124119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.124401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.126600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.143388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:50.144624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:50.213639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a789c4-4e8a-4c17-b622-ff242554b51a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.214284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a789c4-4e8a-4c17-b622-ff242554b51a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.214826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a789c4-4e8a-4c17-b622-ff242554b51a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.216556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 446/911] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:50.222354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.222623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 466/912] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:50.222727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.222981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.240715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:50.241909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:50.245064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:50.326569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 447/913] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:50.348551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:50.358962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c636b-becb-48f2-ab90-a97a76b12464 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:50.376035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:50.377239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:50.401921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:50.403095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.403602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.404154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.405091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:50.407136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:50.428696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 467/914] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:13:50 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:13:50.476969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895edcb-e35c-4a62-b442-e990359999d5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:50.523236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:50.524077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.524403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.524683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.525350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:50.526856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:50.540165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:50.541243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:50.542081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 448/915] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:50.549935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:50.684369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8eda6-7b7d-4689-be37-d622a51a4c45 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:50.722349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:50.723489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.724030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.724515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.725442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:50.725961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:50.727589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:50.733328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 468/916] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:50.740129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:50.824952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:50.845737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d1626-c951-43fb-aedc-c6a48f137131 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:50.883219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:50.883898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:50.884194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:50.884465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:50.885181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:50.885488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:50.886886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7126e-2ea0-44ab-8a69-421a011df4fa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:50.892669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 449/917] 10.222.0.40 () {56 vars in 1114 bytes} [Fri Jul 15 15:13:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:50.898384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:50.940466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:51.007844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8581c4-c18c-420e-bc7f-2ec87bf48e95 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:51.045545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09811a4-edc5-4738-91a6-535a9672bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:51.046215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09811a4-edc5-4738-91a6-535a9672bedf None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:51.046502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09811a4-edc5-4738-91a6-535a9672bedf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:51.046806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09811a4-edc5-4738-91a6-535a9672bedf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:51.047652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09811a4-edc5-4738-91a6-535a9672bedf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:51.049293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09811a4-edc5-4738-91a6-535a9672bedf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:51.072171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 469/918] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:13:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:51.783063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba77d78-e5e3-4443-863c-f650e423e59a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:51.783635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba77d78-e5e3-4443-863c-f650e423e59a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:51.783635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba77d78-e5e3-4443-863c-f650e423e59a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:51.785170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 450/919] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:51.790750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:51.791578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:51.792060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:51.809157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:51.810735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:51.890628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 470/920] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:51.908115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba77d78-e5e3-4443-863c-f650e423e59a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.023237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba77d78-e5e3-4443-863c-f650e423e59a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.058075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.058847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.059127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.059398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.060058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.061477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-50519cf8-81a3-4567-a7d2-cadc1ceafc57 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:52.074109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:52.075187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:52.076362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 451/921] 10.222.0.40 () {56 vars in 1132 bytes} [Fri Jul 15 15:13:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:52.084406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.106284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.162328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393f846-b8e0-4eb5-a263-1cf927d586b6 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.162700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393f846-b8e0-4eb5-a263-1cf927d586b6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.162921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393f846-b8e0-4eb5-a263-1cf927d586b6 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.168204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 452/922] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:52.170037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.170555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.171069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.188908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:52.190505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:52.192970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.199828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df13eec-d1e4-480b-bb03-ee46aa7ece45 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.248993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.250193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.251080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.251754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.252706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:52.253409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.255263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.266364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 471/923] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:52.267909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.278972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 453/924] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:52.301626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.369280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.388972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48462a-904b-46a8-8662-46937f983804 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.429919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81d29d-02d7-4141-933b-d6b0643fce71 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.432228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.433675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.433964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.434240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.434979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:52.435287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.436678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.443048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 472/925] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:52.449746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.473706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.474908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.475411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.475887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.483403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.489596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:52.498289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:52.499280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:13:52.500143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:52.508250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 454/926] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:52.510304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.569253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d9e8a-616a-4b38-b8e7-c52fcd360021 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.606777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347e694-f15c-4ca6-abce-05bafb0a792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.607793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347e694-f15c-4ca6-abce-05bafb0a792d None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.608141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347e694-f15c-4ca6-abce-05bafb0a792d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.608399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347e694-f15c-4ca6-abce-05bafb0a792d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.609160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347e694-f15c-4ca6-abce-05bafb0a792d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.611085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347e694-f15c-4ca6-abce-05bafb0a792d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.617186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e95132-c81e-4f29-af40-559eb131f02c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.642981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 473/927] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:13:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:52.646187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e057c9-e818-4dc2-a1b6-89e4451a8170 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.646851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e057c9-e818-4dc2-a1b6-89e4451a8170 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.647590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e057c9-e818-4dc2-a1b6-89e4451a8170 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.650534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 474/928] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:52.657658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.658393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.659157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.667644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.668682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.669542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.669828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.670161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.671200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:52.671614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.673199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.683460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 455/929] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:52.686753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:52.688466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:52.689091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.768080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.786930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 475/930] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:52.808908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.823535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88534cc9-2a8a-4353-b44d-e5c1f98d6204 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.866851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.867711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.867964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.868258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.880095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.881558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:52.899266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:52.900231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:52.901140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:52.901140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:52.901140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:52.901140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:52.905174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 456/931] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:52.906585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:52.935533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a91a-a908-45fa-9c4f-adfb7f3faba5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:52.973582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:52.974501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:52.974811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:52.975083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:52.975801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:52.977330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:52.979282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:52.994945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:52.996005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:52.996864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 476/932] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:53.002619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.010467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e1b1e-7a5a-489b-9ecc-cea4c3d75350 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.020746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.046391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.047147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.047439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.047809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.048730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:53.048885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.050216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.067396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 457/933] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:53.068864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.114049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c8826-22e0-4c61-877f-c0e544cead69 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.149572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.150478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.150940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.151362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.152220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:53.152684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.154373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6208550-e5da-4ab6-8cce-e8e1b460ed74 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.165020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 477/934] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:53.166897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486203aa-abf8-46eb-9980-d236d698ce2b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.167273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486203aa-abf8-46eb-9980-d236d698ce2b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.167457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486203aa-abf8-46eb-9980-d236d698ce2b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.168877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 478/935] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:53.169443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486203aa-abf8-46eb-9980-d236d698ce2b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.172189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf199f17-ee6a-4366-87de-d03dac5dbc63 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.209847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.210705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.210989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.211258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.218658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.220285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b16f80a0-7e69-4217-9768-57ef9d017ac7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:53.235858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:53.236906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:53.238228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 458/936] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:53.239509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.239735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.240008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.247292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.258097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:53.259314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:53.284786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.290379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486203aa-abf8-46eb-9980-d236d698ce2b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.326530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.327231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.327519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.327793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.328677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:53.329208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.334871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 459/937] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:53.334871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.335087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.340948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 479/938] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:53.349233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.448090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c59f00-e770-4a9c-a553-1c3302cc4937 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.454245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8a13b-cb69-430f-b5f3-43f1a5f5b7fa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.479870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.486287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.486996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.487268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.487548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.488758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:53.489071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.490139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.490507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.490832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.491113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.491386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.492182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] RBAC: Adding query filter params (interface=public, service_id=805d0dffffa141acbbb277cbe67f3bf8, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:53.492490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.493866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.503861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 480/939] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/endpoints?service_id=805d0dffffa141acbbb277cbe67f3bf8&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:53.503861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.513727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 460/940] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:53.514716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.534969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.615965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67c592-229a-46b9-8be9-7b2d59bbc924 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.618565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4afc152-5abe-45c3-82d9-441792960483 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.656052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.656691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.656821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.657080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.657477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.657908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.658175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.658480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.658928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.659603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.659926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.661622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.663212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING keystone.server.flask.common [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 15:13:53.665905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 461/941] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:13:53.666723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.695515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.sql.core [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 15:13:53.695515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 15:13:53.695515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [parameters: {'id': '93ad497ec08b4035be1c8772b1e06ae1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 15:13:53.695515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 93ad497ec08b4035be1c8772b1e06ae1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 93ad497ec08b4035be1c8772b1e06ae1. Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:13:53.704905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 15 15:13:53.706007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application yield resource Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise self.value Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application session.commit() Jul 15 15:13:53.707251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.flush() Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application compat.raise_( Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 15:13:53.708368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 15:13:53.709474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:13:53.710467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [parameters: {'id': '93ad497ec08b4035be1c8772b1e06ae1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.711814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:53.713411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 93ad497ec08b4035be1c8772b1e06ae1. Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 481/942] 10.222.0.40 () {60 vars in 1140 bytes} [Fri Jul 15 15:13:53 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:53.716073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.771560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.786635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:53.815122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97905f59-be7d-4fcb-a0fe-cf52c8179520 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.834459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31646a-9d8b-4a76-99aa-361c031b8542 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:53.858732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.860239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.860687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.861106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.873870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.875471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.877325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:53.877723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:53.878006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:53.879246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:53.885745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:53.887338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:53.896550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:53.897699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:53.898776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:53.898776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:53.898776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:53.898776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:53.898776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 462/943] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:53.904153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:53.913190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:53.914133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:53.915066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 482/944] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:53.922561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:54.013736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d16a3-2e1f-4e37-b57e-aa2caa241491 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:54.037258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c91ca-00b3-4e15-9c14-d8f9efab114e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:54.046471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:54.047234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:54.052032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:54.052848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.052985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.053253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.054050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:54.054287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:54.055767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:54.074990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 463/945] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:54.076474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:54.079545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:54.080432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.080870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.081297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.082209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:54.082719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:54.084329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:54.110712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 483/946] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:54.187390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad43e4-4d98-456e-a96a-6c30f7d4bf86 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:54.225452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:54.226290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.226606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.226887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.234970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:54.236483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:54.252533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:54.253683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:54.254847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:54.258633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 464/947] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:54.260256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:54.374720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75287a75-a607-448f-b2d0-c8bbb148aed1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:54.415731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:54.416455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.416744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.417023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.417830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:54.418142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:54.419649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9890-f6e3-411b-9858-4118781a71af None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:54.437248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 484/948] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:54.438631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:54.494683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:54.542848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce7d07-898a-4a13-b362-33efff7baf71 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:54.578474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920150e-fdeb-428e-90eb-25c83d3350e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:54.579500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920150e-fdeb-428e-90eb-25c83d3350e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.579855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920150e-fdeb-428e-90eb-25c83d3350e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.580206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920150e-fdeb-428e-90eb-25c83d3350e6 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/791ca74ca97b4667a5c170f443982daf/roles/5c1724e5bd744c02830f4aebfeec8458` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.582511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920150e-fdeb-428e-90eb-25c83d3350e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=791ca74ca97b4667a5c170f443982daf, role_id=5c1724e5bd744c02830f4aebfeec8458)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:54.584059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920150e-fdeb-428e-90eb-25c83d3350e6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:54.606714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 465/949] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:13:54 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/791ca74ca97b4667a5c170f443982daf/roles/5c1724e5bd744c02830f4aebfeec8458 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:13:54.802496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ac4e1-c0e1-4625-afce-08acb984a659 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.802933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ac4e1-c0e1-4625-afce-08acb984a659 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.803284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ac4e1-c0e1-4625-afce-08acb984a659 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.804881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 485/950] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:54.810257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:54.810737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:54.811158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:54.828798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:54.830230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:54.903550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 466/951] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:54.925202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ac4e1-c0e1-4625-afce-08acb984a659 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:54.967170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:55.044094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ac4e1-c0e1-4625-afce-08acb984a659 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:55.078819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:55.079872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:55.080298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:55.080714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:55.081521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:55.083085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:55.094974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:55.096207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:13:55.097272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:55.099792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 486/952] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:55.110603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:55.150874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:55.209347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a85c4-a523-4af2-98a6-0d816355d65b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:55.244623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:55.245305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:55.245590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:55.245869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:55.246625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:55.246938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:55.248379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafec6f-0fe7-4189-b54f-8c86a2efdb72 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:55.253886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 467/953] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:55.259506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:55.363504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:55.373902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30970c88-f38a-49ea-bfc9-e034d39d3a2b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:55.409165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:55.409846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:55.410129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:55.410406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:55.411507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:55.411507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:55.412842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ef10c-024d-4cde-9c8e-ebcdd64242ed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:55.418189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 487/954] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:13:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:55.825343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef26c0b-e641-4c7f-bb55-c8b159aa126e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:55.825624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef26c0b-e641-4c7f-bb55-c8b159aa126e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:55.826006 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef26c0b-e641-4c7f-bb55-c8b159aa126e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:55.827698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 468/955] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:55.833451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:55.833746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:55.834033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:55.852106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:55.853649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:55.928920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 488/956] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:13:55.951416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef26c0b-e641-4c7f-bb55-c8b159aa126e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.058355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef26c0b-e641-4c7f-bb55-c8b159aa126e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:56.067012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:56.095051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:56.095792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.096076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.096350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.097012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:56.098387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:56.110695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:56.111848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:56.113168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 469/957] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:56.118285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.206883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:56.222232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1fc84-8799-4438-a87c-1e2aa38fc6fa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:56.262093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:56.262814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.263099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.263380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.264106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:56.264418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:56.265979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458305f-8bee-42e8-acd6-c9be7fcefc0a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:56.271482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 489/958] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:56.277108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.314745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca56749-89a5-48c5-8f76-1a75089efeb4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.315065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca56749-89a5-48c5-8f76-1a75089efeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.315412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca56749-89a5-48c5-8f76-1a75089efeb4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.317181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 490/959] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:56.322218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.322507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.322812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.341551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:56.342899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:56.396426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fee54-e2a5-4289-ab2b-d33c1436a715 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:56.424720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 491/960] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:56.439951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:56.440659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.440955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.441274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.441993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:56.442301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:56.443765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:56.449272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 470/961] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:56.449408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.454919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.502249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:56.543033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:56.567688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aefc01d-2ed2-4567-8ff3-28db7590ac9a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:56.569045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b5406-f536-44e5-9369-104bd5e68900 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:56.606432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:56.606692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa5daf-a35f-4b1d-9536-abe675bb28ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:56.607189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5daf-a35f-4b1d-9536-abe675bb28ee None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.607477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5daf-a35f-4b1d-9536-abe675bb28ee None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.607755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5daf-a35f-4b1d-9536-abe675bb28ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.608075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.608503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.608615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa5daf-a35f-4b1d-9536-abe675bb28ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:56.609008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.610131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:56.610330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa5daf-a35f-4b1d-9536-abe675bb28ee None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:56.612013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:56.628251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.sql.core [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 15:13:56.628251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 15:13:56.628251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [parameters: {'user_id': 'dfa0406822df4350b002255dd7329ff8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 15:13:56.628251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 15:13:56.638174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:13:56.639162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application yield resource Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise self.value Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application session.commit() Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 15:13:56.640171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.flush() Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application compat.raise_( Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 15:13:56.641727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 15:13:56.645637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 15:13:56.647919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dfa0406822df4350b002255dd7329ff8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:56.649811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 15 15:13:56.651790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 492/962] 10.222.0.40 () {60 vars in 1131 bytes} [Fri Jul 15 15:13:56 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:56.654230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.656469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 471/963] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:13:56 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:13:56.769457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17107ca5-a851-434e-8670-7332dad354f9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:56.775456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6171216-3638-4ffc-b072-49dba9d3b36f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.775795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6171216-3638-4ffc-b072-49dba9d3b36f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.776020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6171216-3638-4ffc-b072-49dba9d3b36f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.777712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 472/964] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:56.783765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.784077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.784332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.804180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:56.805398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:56.820815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:56.821695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:56.821985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:56.822258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:56.825748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:56.831157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:56.833436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:56.853178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:56.854184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:56.855208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 493/965] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:13:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:56.860471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.906550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 473/966] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:56.929897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:56.962905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:56.974535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a169021-b167-44d2-b5ec-b849ee94839a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.011691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.012380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.012664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.012942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.013818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:57.014130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:57.015746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:57.049737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66686bca-5449-40c9-9e59-e83448da4b68 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.058688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 494/967] 10.222.0.40 () {56 vars in 1101 bytes} [Fri Jul 15 15:13:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:57.089695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.090964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.091452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.092047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.104662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:13:57.113994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:57.115082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:13:57.116010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:57.119011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 474/968] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Jul 15 15:13:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:57.125151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:57.193817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:57.243648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88daf7-8ecd-4013-830e-596db54d20f1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.282095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.282805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.283094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.283372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.284134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:57.284442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:57.285837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:57.295207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 495/969] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:13:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:57.301108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:57.396018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:57.418071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9bead-dc62-4949-9cac-336420b4408c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.454310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.455174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.455461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.455739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.466668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:57.468126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-db8a99d0-9b87-4d04-be1e-85c54cd97eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:57.485674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:57.486641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:57.487928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:57.487928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:57.487928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:57.487928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:57.487928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 475/970] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:57.494078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:57.542338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e2906-1907-4938-9bb8-969cca058e0e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.542738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e2906-1907-4938-9bb8-969cca058e0e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.543000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e2906-1907-4938-9bb8-969cca058e0e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.544325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 476/971] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:57.549228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.549511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.549779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.568029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:57.569218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:57.598422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c264ce0-10a3-4a84-886c-7ef877258cea None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.615647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:57.635159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.635835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.636120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.636398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.637126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:57.637435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:57.638856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:57.640778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 477/972] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:57.654814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 496/973] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:57.660564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:57.664056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:57.724758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:57.764214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb73351-be4e-4756-bc9f-febd084501ac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.769837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1268d-8819-4bbb-8584-8d0e7576ebd5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.799567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.800397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.800680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.800958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.808110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:57.808561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:57.809426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:57.809930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:57.810145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:57.810727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:57.811781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:57.814112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:13:57.826772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:57.827917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:57.829031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 478/974] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:13:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:57.833908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:57.834981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 497/975] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:13:57 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:13:57.954863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac026bdb-b360-41b2-9e58-a4a5f37f2b66 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:57.980550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:58.004540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:58.005212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.005490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.005758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.006614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:58.006941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:58.008337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:58.027313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 479/976] 10.222.0.40 () {56 vars in 1149 bytes} [Fri Jul 15 15:13:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:58.033481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:58.071393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:58.139572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795292-089b-472b-89a6-eb001f6d19bc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:58.174840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:58.175501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.175780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.176050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.244876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:58.246277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:58.279639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:58.347313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 498/977] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:13:58 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:13:58.352888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:58.452964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25497996-6f4f-4e7b-adb2-9cf97c3794b1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:58.488159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:58.488973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.489253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.489528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.500616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:58.502195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:13:58.520241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:58.521193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:58.522113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:13:58.522113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:13:58.522113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:13:58.522113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:58.526546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 480/978] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:58.527444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:58.633469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc26649-9fc8-46e2-a181-9c0b299b19fc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:58.668163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:58.668829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.669109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.669385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.670106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:58.670414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:58.671883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa976b9-fff6-46ad-bd86-369161460f7e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:58.674324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:58.685694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 499/979] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:58.691526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:58.721142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:58.727608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1d3f1-6894-4db3-ba70-2ba220768b55 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.727962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1d3f1-6894-4db3-ba70-2ba220768b55 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.728334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1d3f1-6894-4db3-ba70-2ba220768b55 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.730033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 500/980] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:13:58.734866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.735151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.735433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.752617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:58.754160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:13:58.802169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de93370-fb65-4f05-a5ed-df935a99d343 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:58.825890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 501/981] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:13:58.838715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:58.839593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:58.839945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:58.840297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:58.847250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:58.848233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:58.848794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:13:58.863826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:58.864933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:13:58.865918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 481/982] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:13:58.871334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:58.962778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a882e6-85c0-4375-9722-a8709d618211 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:58.975930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:58.989769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefac884-b066-470e-940a-2e18fda942a6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:58.990829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:59.001162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.002118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.002494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.002942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.003665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.005160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:13:59.017321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:13:59.018551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:13:59.019618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:13:59.020708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 502/983] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:13:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:13:59.026178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:59.031782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.032725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.033007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.033262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.034077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:59.034385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.036012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.049078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 482/984] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:59.055037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:59.139073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc286b19-c7ad-49bb-b7a6-51082afcd25f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:59.166317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074f93b-ded2-4a4d-9b03-a8328e895c23 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:59.177098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.177777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.178462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.178772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.179798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:59.180107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.182224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.190494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 503/985] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:59.194068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:59.210116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.210823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.211117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.211389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.212218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] RBAC: Adding query filter params (role.id=5c1724e5bd744c02830f4aebfeec8458, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=791ca74ca97b4667a5c170f443982daf) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:59.212532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.213971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.230560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 483/986] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:13:59 2022] GET /identity/v3/role_assignments?user.id=791ca74ca97b4667a5c170f443982daf&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=5c1724e5bd744c02830f4aebfeec8458 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:13:59.269942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:59.327193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b22556-9cf3-4024-a170-7c660c4f9f08 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:59.364208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.364880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.365155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.365424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.366137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:59.366438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.367904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.373490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 504/987] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:13:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:59.380348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:59.397049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:59.496359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ab40c-a13d-4b3d-b38f-824cc62037ba None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:59.532580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.533181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.533462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.533730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.534540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] RBAC: Adding query filter params (interface=admin, service_id=805d0dffffa141acbbb277cbe67f3bf8, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:13:59.534861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.536223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70f9ae-7773-4fe3-8bdc-6ce95d5e7b1b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.541681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 484/988] 10.222.0.40 () {56 vars in 1249 bytes} [Fri Jul 15 15:13:59 2022] GET /identity/v3/endpoints?service_id=805d0dffffa141acbbb277cbe67f3bf8&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:13:59.547477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:13:59.670571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeaba41-e6bf-4315-b3f5-1af62ced451e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:13:59.694239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:13:59.718019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075938f-85a9-432a-931d-08720a5359db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:13:59.718954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075938f-85a9-432a-931d-08720a5359db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.719222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075938f-85a9-432a-931d-08720a5359db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.719501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075938f-85a9-432a-931d-08720a5359db None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.720182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075938f-85a9-432a-931d-08720a5359db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:13:59.721557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075938f-85a9-432a-931d-08720a5359db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:13:59.734583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 505/989] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:13:59 2022] GET /identity/v3/services => generated 1727 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:13:59.892725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e795d-b4d5-4ed2-b7e4-6a326c20a4e9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.893034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e795d-b4d5-4ed2-b7e4-6a326c20a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.893236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e795d-b4d5-4ed2-b7e4-6a326c20a4e9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.894782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 485/990] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:13:59.900374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:13:59.900656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:13:59.900934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:13:59.919296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:13:59.920481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:00.012469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 506/991] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:13:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:00.035756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e795d-b4d5-4ed2-b7e4-6a326c20a4e9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:00.126786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:00.154310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e795d-b4d5-4ed2-b7e4-6a326c20a4e9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:00.193682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:00.194696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:00.194982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:00.195251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:00.195921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:00.197298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:00.210046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:00.211381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:14:00.212227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:00.214926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 486/992] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:14:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:00.217483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:00.335256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e7653-999a-4e59-9381-ba7f77647eb1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:00.351082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:00.377305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:00.377996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:00.378276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:00.378560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:00.379304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:00.379606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:00.381095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:00.387434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 507/993] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:14:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:00.394352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:00.522894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d4984-fc2e-40b6-943e-d21b4e1c4f25 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:00.570354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:00.571076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:00.571355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:00.571625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:00.572333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:00.572634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:00.574001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:00.585134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:00.585567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 487/994] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:14:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:00.711101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3deafc9-0a9c-44d7-b59c-0291f73b95d7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:00.746483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:00.747184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:00.747464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:00.747736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:00.748530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] RBAC: Adding query filter params (interface=public, service_id=e5f38b39bc3142cf8079b92649d99734, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:00.748842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:00.750343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09fcbf-328b-4f39-aa4c-34522873ffee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:00.755939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 508/995] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:14:00 2022] GET /identity/v3/endpoints?service_id=e5f38b39bc3142cf8079b92649d99734&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:00.761737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:00.766178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:00.879758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2eb2af-4426-4cb7-b38d-0faf7c10416c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:00.923358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5d854-10d5-48c9-a2f3-83995d110715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:00.924218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5d854-10d5-48c9-a2f3-83995d110715 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:00.924501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5d854-10d5-48c9-a2f3-83995d110715 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:00.924775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5d854-10d5-48c9-a2f3-83995d110715 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:00.925466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5d854-10d5-48c9-a2f3-83995d110715 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:00.926871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5d854-10d5-48c9-a2f3-83995d110715 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:00.934021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 488/996] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:14:00 2022] GET /identity/v3/services => generated 1727 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:14:01.309861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcfd2-184b-481d-8b7b-c7aa225a96b2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.310418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcfd2-184b-481d-8b7b-c7aa225a96b2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.310418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdcfd2-184b-481d-8b7b-c7aa225a96b2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.311808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 509/997] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:01.317764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800824f-a706-4349-ac21-538739aef957 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.318192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800824f-a706-4349-ac21-538739aef957 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.318369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800824f-a706-4349-ac21-538739aef957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.335897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1800824f-a706-4349-ac21-538739aef957 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:01.337093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1800824f-a706-4349-ac21-538739aef957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:01.405722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:01.412224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 489/998] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:01.433735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdcfd2-184b-481d-8b7b-c7aa225a96b2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:01.480140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:01.541069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdcfd2-184b-481d-8b7b-c7aa225a96b2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:01.577474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:01.578347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.578652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.578923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.579573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:01.581009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:01.593078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:01.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:01.595146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:01.598863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 510/999] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Jul 15 15:14:01 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:01.600500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800824f-a706-4349-ac21-538739aef957 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:01.703700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800824f-a706-4349-ac21-538739aef957 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:01.739632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:01.740298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.740574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.740846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.741545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:01.741844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:01.743251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5aefb-8a44-44c5-89e6-0d117b8792a9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:01.748280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 490/1000] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:01.754432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:01.784461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892c49c-7185-4cb6-b787-24c559983c86 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.784769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892c49c-7185-4cb6-b787-24c559983c86 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.785041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892c49c-7185-4cb6-b787-24c559983c86 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.786471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 491/1001] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:01.791542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.791832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.792114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.809503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:01.810675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:01.859613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb97dd-ed54-4e77-a75f-f33475af82a0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:01.863357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:01.873663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:01.886486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 492/1002] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:01.896652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:01.897318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:01.897593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:01.897863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:01.898590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:01.898897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:01.900280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:01.905264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 511/1003] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:01.909389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:02.033575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e26640-fdeb-4cde-a28f-02a53fc621f1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:02.072091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:02.072912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:02.073186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:02.073456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:02.074119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:02.075551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:02.088415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:02.089446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:02.090361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 493/1004] 10.222.0.40 () {56 vars in 1126 bytes} [Fri Jul 15 15:14:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:02.096005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:02.202461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706bd1e-3d9c-4a03-900c-eea4a3845190 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:02.236665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:02.237733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:02.238012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:02.238284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:02.239004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:02.239314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:02.240689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:02.245907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 512/1005] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:14:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:02.252106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:02.308860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:02.355767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274a2d1-2308-4f3b-a092-76a6096987fa None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:02.395227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:02.395933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:02.396218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:02.396494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:02.397209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:02.397521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:02.398974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b44bf9-0655-4c9c-b3ff-a9f82be2a88c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:02.404385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 494/1006] 10.222.0.40 () {56 vars in 1122 bytes} [Fri Jul 15 15:14:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:02.410496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:02.445774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:02.517752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90db4e-38e9-40ac-a0f4-ac03f6864247 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:02.553202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b14b80-e00e-4e1f-9b97-35886286683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:02.553870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b14b80-e00e-4e1f-9b97-35886286683d None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:02.554152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b14b80-e00e-4e1f-9b97-35886286683d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:02.554428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b14b80-e00e-4e1f-9b97-35886286683d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:02.555208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b14b80-e00e-4e1f-9b97-35886286683d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:02.556611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b14b80-e00e-4e1f-9b97-35886286683d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:02.578570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 513/1007] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:14:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:02.974733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4c23d-c10b-465c-be55-a107e7fe9ae8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:02.975023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4c23d-c10b-465c-be55-a107e7fe9ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:02.975303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4c23d-c10b-465c-be55-a107e7fe9ae8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:02.976697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 495/1008] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:02.982078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:02.982374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:02.982694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.003369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:03.004575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:03.092090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 514/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:03.114810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4c23d-c10b-465c-be55-a107e7fe9ae8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:03.220022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4c23d-c10b-465c-be55-a107e7fe9ae8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:03.240612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:03.255965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:03.256799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:03.257089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:03.257341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.258002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:03.259410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:03.270967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:03.271945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:03.272857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 496/1010] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:14:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:03.278175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:03.300352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:03.388651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f67a6-5d54-44bb-a80e-f81921f9be62 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:03.424113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:03.424851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:03.425037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:03.425311 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.426014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:03.426317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:03.427756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:03.437547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 515/1011] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:14:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:03.439264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:03.545236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643bc54-b5dc-4628-be6d-b61ad002d676 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:03.579546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:03.580222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:03.580504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:03.580782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.581485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:03.581791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:03.583196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab79ae9-65b8-43b0-a276-b104450ed7c1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:03.588396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 497/1012] 10.222.0.40 () {56 vars in 1112 bytes} [Fri Jul 15 15:14:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:03.594187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:03.677842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:03.699267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8dcf9-b0fd-4ffa-ac75-50c8b48a606c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:03.734323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8282484-c3df-40c8-97e5-8de29bb94b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:03.735399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8282484-c3df-40c8-97e5-8de29bb94b0c None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:03.735689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8282484-c3df-40c8-97e5-8de29bb94b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:03.735968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8282484-c3df-40c8-97e5-8de29bb94b0c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.736749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8282484-c3df-40c8-97e5-8de29bb94b0c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:03.738539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8282484-c3df-40c8-97e5-8de29bb94b0c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:03.766560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 516/1013] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:14:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:03.947220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f92da-3fce-4a3e-a35a-76a6cf65caa5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:03.947514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f92da-3fce-4a3e-a35a-76a6cf65caa5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:03.947794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f92da-3fce-4a3e-a35a-76a6cf65caa5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.949311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 498/1014] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:03.955355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca019-7464-429c-8a38-1e735253bfd4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:03.955644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca019-7464-429c-8a38-1e735253bfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:03.955924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ca019-7464-429c-8a38-1e735253bfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:03.973971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e99ca019-7464-429c-8a38-1e735253bfd4 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:03.975233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e99ca019-7464-429c-8a38-1e735253bfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:04.053671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 517/1015] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:04.075721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f92da-3fce-4a3e-a35a-76a6cf65caa5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:04.129193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:04.183308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f92da-3fce-4a3e-a35a-76a6cf65caa5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:04.218239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b0ce-66c3-40e8-b6ad-5b947ad60f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:04.219027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b0ce-66c3-40e8-b6ad-5b947ad60f14 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:04.219307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b0ce-66c3-40e8-b6ad-5b947ad60f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:04.219588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b0ce-66c3-40e8-b6ad-5b947ad60f14 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:04.220349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6b0ce-66c3-40e8-b6ad-5b947ad60f14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:04.221737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6b0ce-66c3-40e8-b6ad-5b947ad60f14 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:04.238418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 499/1016] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:14:04 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:04.641864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771975a-0041-44bf-a5ca-081ee9e08946 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:04.642388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771975a-0041-44bf-a5ca-081ee9e08946 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:04.642388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771975a-0041-44bf-a5ca-081ee9e08946 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:04.643983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 518/1017] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:04.649870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:04.650304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:04.650409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:04.667963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:04.669186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:04.748564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 500/1018] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:04.771338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771975a-0041-44bf-a5ca-081ee9e08946 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:04.838495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:04.880430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771975a-0041-44bf-a5ca-081ee9e08946 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:04.917225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4eab9b-1de9-443e-a6d0-079ca25cc001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:04.917953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eab9b-1de9-443e-a6d0-079ca25cc001 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:04.918224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eab9b-1de9-443e-a6d0-079ca25cc001 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:04.918494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4eab9b-1de9-443e-a6d0-079ca25cc001 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:04.919710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4eab9b-1de9-443e-a6d0-079ca25cc001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:04.921280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4eab9b-1de9-443e-a6d0-079ca25cc001 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:04.923049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 519/1019] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:04.928960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:04.963367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:05.032797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7746319f-7537-4051-9a90-514c5d8711fe None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:05.069545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3589a4-4a2e-4443-ae0c-95c68a053d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:05.070210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3589a4-4a2e-4443-ae0c-95c68a053d05 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:05.070473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3589a4-4a2e-4443-ae0c-95c68a053d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:05.070799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3589a4-4a2e-4443-ae0c-95c68a053d05 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:05.071503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3589a4-4a2e-4443-ae0c-95c68a053d05 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:05.072938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3589a4-4a2e-4443-ae0c-95c68a053d05 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:05.098747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 501/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:14:04 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:05.881304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ea86-2ce8-49ad-8614-4301707402ea None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:05.881888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ea86-2ce8-49ad-8614-4301707402ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:05.881888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2ea86-2ce8-49ad-8614-4301707402ea None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:05.883585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 520/1021] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:05.888760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:05.889190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:05.889297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:05.907139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:05.908344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:05.982581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 502/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:06.006558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ea86-2ce8-49ad-8614-4301707402ea None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.110422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2ea86-2ce8-49ad-8614-4301707402ea None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.141700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:06.148053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e88ad-7e68-4a8c-aadf-b56572da1921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.148889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e88ad-7e68-4a8c-aadf-b56572da1921 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.149167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e88ad-7e68-4a8c-aadf-b56572da1921 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.149440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e88ad-7e68-4a8c-aadf-b56572da1921 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.160574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e88ad-7e68-4a8c-aadf-b56572da1921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9d0e88ad-7e68-4a8c-aadf-b56572da1921 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:06.168709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:06.169661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:14:06.170557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:06.172177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 521/1023] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:06.176739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.195879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:06.279965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3604a-6b29-4838-afd5-ceeb3767066f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.280244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3604a-6b29-4838-afd5-ceeb3767066f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.280584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3604a-6b29-4838-afd5-ceeb3767066f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.281977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 522/1024] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:06.284912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be05c0f-bcc0-4752-b1b0-d6901ffe20d2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.286792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.287083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.287364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.304319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:06.305534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:06.325659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.326342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.326666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.326946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.328099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:06.328884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.330588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.339839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 503/1025] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:06.346940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.380935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 523/1026] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:14:06.405363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.428287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:06.452457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4362a3-8702-47d2-ae7c-3d075c5af67e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.489478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.490276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.490754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.491031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.501767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.503229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.518757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13698641-7ff5-4e46-84dd-82bea1faf545 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:06.520693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:06.521577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:06.522491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:06.522491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:06.522491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:06.522491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:06.522491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 504/1027] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:06.527708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.546409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:06.559111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.560006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.560350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.560688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.561352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.562988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:06.574868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:06.575914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:06.576778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 524/1028] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:06.582384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.637938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a822-ce09-4424-a105-d5361ad8efbb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.667994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:06.675657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.676327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.676610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.676882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.677609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:06.677912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.679302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.689515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33509d05-a8a7-47a2-bfbe-d17ca549a188 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.691762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 505/1029] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:06.697449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.724673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.725537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.725921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.726202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.726960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] RBAC: Adding query filter params (name=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:06.727281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.728660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.735657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 525/1030] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:06.741912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.800211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:06.817817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f452b3-05ed-4ed3-8450-da9b09c2d1d1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.855508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcbd0b-6b27-447a-9066-62855fefba1c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:06.859055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.860069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.860522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.861069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.869648 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.871197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:14:06.886892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:06.887882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:06.889973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 506/1031] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:06.893896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.895621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:06.896306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:06.896615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:06.896853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:06.897567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:06.897930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:06.899388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:06.904679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 526/1032] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:06.910873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:06.950843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.000831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846e4c2-634e-45ea-a1d4-1d5056843eb6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.027559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db486f3-5292-4dd7-86a8-4571aaaa3993 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.039462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.040141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.040514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.040801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.041438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.042333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:07.042672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.044261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.063795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.064464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.064748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.065011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.065828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] RBAC: Adding query filter params (interface=public, service_id=8e4d35330f6740c094d8bb4f30f0dc55, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:07.066143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.067569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.073304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 507/1033] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:07.074668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 527/1034] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:14:06 2022] GET /identity/v3/endpoints?service_id=8e4d35330f6740c094d8bb4f30f0dc55&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:07.074793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.079776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.186008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0351f7-1cc9-4c3c-84b9-01e60139c72a None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.186333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4908f82-be46-488b-8bb5-4da786f21934 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.195484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.220708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab52309-06ed-43ea-a229-244e22c14070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.221376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab52309-06ed-43ea-a229-244e22c14070 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.221657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab52309-06ed-43ea-a229-244e22c14070 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.221929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab52309-06ed-43ea-a229-244e22c14070 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.222622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab52309-06ed-43ea-a229-244e22c14070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.222839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.223490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.223765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.224011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab52309-06ed-43ea-a229-244e22c14070 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.224117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.230478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 528/1035] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:14:07.231783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335d85-8458-46b6-9aa6-a36ce18b9833 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.232056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335d85-8458-46b6-9aa6-a36ce18b9833 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.232349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d335d85-8458-46b6-9aa6-a36ce18b9833 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.233734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 529/1036] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:07.240076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.240391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.240646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.258790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:07.260277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:07.301624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.305131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.306634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.339302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 530/1037] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:14:07.368797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.426621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 508/1038] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:07.432625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.442707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.477448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb109cbb-e3e5-4ae8-8495-14354ba9a46f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.513921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.514801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.515159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.515533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.516890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.518341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.521528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 531/1039] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:07.527359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.546212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a55e4-81e0-45fc-8e19-d11b0bbe90ec None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.573279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.583716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.584536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.584829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.585112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.596180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.597698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:07.616626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:07.617604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:07.618594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:07.618594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:07.618594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:07.618594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:07.618594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 509/1040] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:07.623513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.641824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b11507-dab1-4938-9abe-bfe81315958d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.681540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.682588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.683453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.683905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.685066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.687618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.689455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.714363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 532/1041] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:14:07 2022] POST /identity/v3/users => generated 285 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:14:07.740710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b27d89-815a-417e-98df-c81f090d3ee9 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.779814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.780593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.780870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.781144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.781885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:07.782192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.783647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.796824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 510/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:07.802745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:07.905893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:07.912940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e97b4-82cc-4b7b-8b2a-36487871602c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:07.950574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:07.951387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:07.951670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:07.951941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:07.960231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:07.961714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:14:07.976069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:07.976985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:07.977917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 533/1043] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:07.983558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:08.087028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25e248c-3c4c-4377-b81e-6de68482591e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:08.106793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:08.123218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:08.123877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:08.124220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:08.124498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:08.125290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:08.125596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:08.127068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e7ca6-3526-4d2e-9c75-f1bbf55bd1e0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:08.139781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 511/1044] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:14:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:08.145781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:08.258164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfa48b-2d40-4415-ab74-c88b40f227c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:08.295106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:08.296440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:08.296939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:08.297414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:08.298458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] RBAC: Adding query filter params (role.id=5c1724e5bd744c02830f4aebfeec8458, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=063c589447364966b64fcc2da5c1aa02) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:08.299040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:08.300698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8507dfc-ace3-4237-bb00-208d42a0156c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:08.307384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 534/1045] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:08 2022] GET /identity/v3/role_assignments?user.id=063c589447364966b64fcc2da5c1aa02&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=5c1724e5bd744c02830f4aebfeec8458 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:09.293011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd070c4-3f14-4bc0-b035-961f07842826 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.293566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd070c4-3f14-4bc0-b035-961f07842826 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.293566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd070c4-3f14-4bc0-b035-961f07842826 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.294936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 512/1046] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:09.301130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.301572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.301663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.319547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:09.320759 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:09.336799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:09.395308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 535/1047] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:09.417040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd070c4-3f14-4bc0-b035-961f07842826 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:09.533214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd070c4-3f14-4bc0-b035-961f07842826 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:09.569200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:09.569925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.570205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.570483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] PATH_INFO: `/v3/services/image` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.571248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:09.572622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:09.584663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:09.585666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:09.591624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 513/1048] 10.222.0.40 () {56 vars in 1105 bytes} [Fri Jul 15 15:14:09 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:09.594227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:09.697866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b5ef0-7e7d-4035-9a59-1ea49582bc34 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:09.733310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:09.734306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.734615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.734894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.735614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] RBAC: Adding query filter params (name=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:09.735921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:09.737684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b325295-4fc7-4375-a0f7-fb76eb14ff4b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:09.747732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 536/1049] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:09.748896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:09.800588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:09.807549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44d9e5-e3ee-447c-b7f5-4451ea72d790 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.807844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44d9e5-e3ee-447c-b7f5-4451ea72d790 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.808107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44d9e5-e3ee-447c-b7f5-4451ea72d790 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.809498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 537/1050] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:14:09.815229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.815516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.815836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.833525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:09.834677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:09.846328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:09.862328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8a289-c889-4bd7-b8ed-3e354b91740c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:09.899682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:09.900682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:09.900967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:09.901240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:09.901959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] RBAC: Adding query filter params (type=image) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:09.902274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:09.903746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:09.913721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 514/1051] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:09.913887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 538/1052] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:09.915620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:09.933581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.030578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4177094c-84ad-406f-aea0-a6bdf3917004 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.040530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.047001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6578b-51a7-4057-ac5b-bba49dc5d1d9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.069559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.070270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.070509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.070811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.071682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.072919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.083137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.084144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.084331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.084607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.093338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 515/1053] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:14:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:10.095780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:10.103691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:10.104725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:10.105617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:10.111022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 539/1054] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:10.112423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.239609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d364c0-82d7-45a3-80f7-d68840489504 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.276867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.277540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.277823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.278108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.278892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:10.279202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.280647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb8eee-006d-4b88-b217-81027de4b177 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.289327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 516/1055] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:10.295303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.299748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.333257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f00ae7-b479-417b-a5af-52ca05f07186 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.333822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f00ae7-b479-417b-a5af-52ca05f07186 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.333822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f00ae7-b479-417b-a5af-52ca05f07186 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.335256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 517/1056] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:14:10.340216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67912419-6c45-4150-bb50-dc7108db738a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.340500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67912419-6c45-4150-bb50-dc7108db738a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.340774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67912419-6c45-4150-bb50-dc7108db738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.358250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-67912419-6c45-4150-bb50-dc7108db738a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:10.359608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-67912419-6c45-4150-bb50-dc7108db738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:10.378012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.407282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfb387-0701-4542-9e54-ff899516c493 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.433434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 518/1057] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:14:10.444504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.445408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.445688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.445959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] PATH_INFO: `/v3/users/e0b5500218594d83a8ed1929920095f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.455944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67912419-6c45-4150-bb50-dc7108db738a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.462673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b5500218594d83a8ed1929920095f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.464572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.471542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 540/1058] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/users/e0b5500218594d83a8ed1929920095f0 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:10.473023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.536268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.560764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67912419-6c45-4150-bb50-dc7108db738a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.578181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e1303-2451-4a78-ba96-59af08fab1d2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.595551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.596298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.596570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.596845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.607775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:10.615267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.615924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.616204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.616476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:10.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:10.617560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:10.619104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 519/1059] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:10.624227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.640167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.691243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.692725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.730795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778d132-324e-4446-9c29-c455283e9486 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.758167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.767395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.768079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.768363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.768640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.769394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:10.769706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.771121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.779349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 520/1060] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:10.785211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.813951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 541/1061] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:10.815446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.861885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:10.887147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119316-dc7c-49de-be8e-3a68ec77f045 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.922752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd9c89-6b3d-4997-b5e6-f2bcfda53e3a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:10.923448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.924344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.924580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.924869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.935471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.937630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:10.958991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:10.960011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:10.961036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:10.961036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:10.961036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:10.961036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:10.961036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 521/1062] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:10.966050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:10.971656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:10.972838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:10.973334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:10.973811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:10.975197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9a637a1bbc4e629d7196773078e709)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:10.976875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:10.978859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 542/1063] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:10.985514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:11.019737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:11.077872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b9f87-eb46-4e26-9c07-9546fa937e3f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.091411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6316a6-70f1-44dd-a220-b2bdf1cfd691 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.107222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:11.117948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.118490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.118809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.119090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.119820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:11.120137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.121589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:11.133129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.133859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 522/1064] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:11.134217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.134504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.134813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.135725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=bd9a637a1bbc4e629d7196773078e709, user.id=e0b5500218594d83a8ed1929920095f0) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:11.136037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.137482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:11.139391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:11.146866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 543/1065] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:10 2022] GET /identity/v3/role_assignments?user.id=e0b5500218594d83a8ed1929920095f0&scope.project.id=bd9a637a1bbc4e629d7196773078e709&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:11.264399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48adde-26b1-411f-ba9a-c3ed94529de4 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.303039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.303849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.304120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.304397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.312433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.313887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:14:11.329044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:11.330034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:11.331394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 523/1066] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:14:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:11.336152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:11.434229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:11.443054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729836b8-ed25-47e3-b385-46c1433aa872 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.477276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.477953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.478236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.478508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.479376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:11.479681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.481120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:11.500302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 544/1067] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:14:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:11.505944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:11.555123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:11.610662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba602-0a88-4706-924a-2a0169e48d57 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.646961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.647846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.648238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.648520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.659553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.661010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:11.679199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:11.680167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:11.681071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:11.681071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:11.681071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:11.681071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:11.682912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 524/1068] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:11.686571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:11.791156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5528f0-386d-495a-b61e-4081e5263b24 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.826126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.826946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.827229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.827508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.828258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:11.828563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.829960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:11.842273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 545/1069] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:11.848253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:11.946753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:11.951664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e111ba-f891-47df-9dbf-f4033c7e39e1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:11.988414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:11.989225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:11.989508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:11.989786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:11.996762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:11.998223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-33bfbfa9-e0e0-4ef0-be72-bc2e82969e92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:14:12.012709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:12.013600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:12.019825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 525/1070] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:14:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:12.020320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:12.036856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:12.127731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871ea8f-5a7f-4f44-a1c5-4909f78f8f37 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:12.134235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd8391-c33e-4211-9e79-dddbf4caae14 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.134545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd8391-c33e-4211-9e79-dddbf4caae14 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.134992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd8391-c33e-4211-9e79-dddbf4caae14 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.136376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 526/1071] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:12.141755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.142144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.142498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.161168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:12.162699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:12.165360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:12.166062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.166324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.166624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.167414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:12.167724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:12.169158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:12.181557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 546/1072] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:12.187682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:12.242248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 527/1073] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:12.264905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:12.269975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:12.303040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0f50a-3af0-4c06-95b1-ed9134d2b208 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:12.341068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:12.342030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.342350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.342646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/063c589447364966b64fcc2da5c1aa02/roles/5c1724e5bd744c02830f4aebfeec8458` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.345041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=063c589447364966b64fcc2da5c1aa02, role_id=5c1724e5bd744c02830f4aebfeec8458)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:12.346485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:12.352835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:12.362396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 547/1074] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:14:12 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/063c589447364966b64fcc2da5c1aa02/roles/5c1724e5bd744c02830f4aebfeec8458 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:14:12.366482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c684f4d-3b54-43d9-bb21-c3b1d9b63f96 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:12.403024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:12.403692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.403968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.404243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.479228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:12.480727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:12.578232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:12.591721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 528/1075] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 15:14:12.597956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:12.703387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4dfd58-2579-4922-b453-398706a2f11b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:12.733185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:12.739782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:12.740603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.740893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.741170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.752841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:12.754316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-26fe48ba-c138-4913-946c-1197958dc806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:12.772184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:12.773104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:12.774018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:12.774018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:12.774018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:12.774018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:12.778377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 548/1076] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:12.779600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:12.883431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b2a37-cae3-44c8-aae7-ecbde92ba5a0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:12.918808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:12.919485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:12.919769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:12.920039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:12.921037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:12.921217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:12.922626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e7bef-eb45-47c6-b20d-c97346843310 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:12.934632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 529/1077] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:13.151410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70430db2-6188-46bb-af7e-57936bc39bf1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:13.151601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70430db2-6188-46bb-af7e-57936bc39bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:13.151854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70430db2-6188-46bb-af7e-57936bc39bf1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:13.153258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 549/1078] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:13.159010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:13.159310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:13.159605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:13.177305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:13.178541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:13.227247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:13.252111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 530/1079] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:13.273934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70430db2-6188-46bb-af7e-57936bc39bf1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:13.377159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70430db2-6188-46bb-af7e-57936bc39bf1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:13.411705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:13.412415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:13.412692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:13.412961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:13.423162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:13.430810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:13.431789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:13.432924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 550/1080] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:13.438854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:13.541516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8ec1c-4e77-43a0-b165-ac6eeb8f03c8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:13.577945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:13.578646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:13.579210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:13.579210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:13.579910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:13.580213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:13.581575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:13.589748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 531/1081] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:13.595886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:13.644078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:13.701053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b370-c055-4d0f-94fa-5e8cc34b9d62 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:13.734188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:13.735098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:13.735375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:13.735640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] PATH_INFO: `/v3/users/e0b5500218594d83a8ed1929920095f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:13.737184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b5500218594d83a8ed1929920095f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:13.738654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:13.740277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 551/1082] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:13 2022] GET /identity/v3/users/e0b5500218594d83a8ed1929920095f0 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:13.746189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:13.789167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:13.850666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f5cf0-cd89-4630-9151-328647e39caf None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:13.884686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65745432-ff21-483d-95cc-520eb284e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:13.885545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745432-ff21-483d-95cc-520eb284e814 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:13.885820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745432-ff21-483d-95cc-520eb284e814 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:13.886088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65745432-ff21-483d-95cc-520eb284e814 None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:13.887303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65745432-ff21-483d-95cc-520eb284e814 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9a637a1bbc4e629d7196773078e709)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:13.888728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65745432-ff21-483d-95cc-520eb284e814 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:13.890985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 532/1083] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:13 2022] GET /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:13.896999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:14.004543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5002cd-bacf-4b70-ab57-eba4eea14d62 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:14.017865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:14.047452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c6231-67cc-4c10-b2a0-f8c317c05c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:14.048400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6231-67cc-4c10-b2a0-f8c317c05c55 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:14.048634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6231-67cc-4c10-b2a0-f8c317c05c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:14.048946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c6231-67cc-4c10-b2a0-f8c317c05c55 None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709/users/e0b5500218594d83a8ed1929920095f0/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:14.051074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6231-67cc-4c10-b2a0-f8c317c05c55 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9a637a1bbc4e629d7196773078e709, user_id=e0b5500218594d83a8ed1929920095f0, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:14.052535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c6231-67cc-4c10-b2a0-f8c317c05c55 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:14.066310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 552/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:14:13 2022] PUT /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709/users/e0b5500218594d83a8ed1929920095f0/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:14:14.398097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e636d5-4c9d-40c0-900a-0b4d8077f124 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:14.398297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e636d5-4c9d-40c0-900a-0b4d8077f124 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:14.398554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e636d5-4c9d-40c0-900a-0b4d8077f124 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:14.399954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 533/1085] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:14.405737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:14.406022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:14.406292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:14.423481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:14.424657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:14.494544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 553/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:14.520513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e636d5-4c9d-40c0-900a-0b4d8077f124 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:14.625764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e636d5-4c9d-40c0-900a-0b4d8077f124 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:14.634939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:14.661050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:14.661786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:14.662061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:14.662327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:14.672624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:14.680800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:14.681818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:14.682709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 534/1087] 10.222.0.40 () {56 vars in 1120 bytes} [Fri Jul 15 15:14:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:14.688629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:14.789430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b61192-eaa5-4cde-882b-86e41103c32c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:14.791037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:14.832889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:14.833558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:14.833904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:14.834383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:14.835841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:14.836316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:14.838537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:14.847550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 554/1088] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:14:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:14.853302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:14.958462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910830b4-9d4c-4883-aa5d-6452d827b88f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:14.993226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:14.994017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:14.994291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:14.994587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:15.005269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:15.006736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:15.023739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:15.024658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:15.025596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:15.025596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:15.025596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:15.025596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:15.025596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 535/1089] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:15.030948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:15.136439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e813ba0-ca76-490e-ad2a-0e6dc7a5d4ad None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:15.174037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:15.174745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:15.175032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:15.175290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:15.176050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:15.176357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:15.177779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:15.191476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 555/1090] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:15.197543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:15.221318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:15.305030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bb787-e5f8-43ac-9616-51dbc297d0ab None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:15.340114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:15.340915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:15.341200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:15.341474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:15.349355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:15.350818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:14:15.365532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:15.366565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:15.367560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 536/1091] 10.222.0.40 () {56 vars in 1093 bytes} [Fri Jul 15 15:14:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:15.372937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:15.400443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:15.479076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6ed0e-12e8-4e80-9967-66428c94594d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:15.515404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:15.516076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:15.516358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:15.516635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:15.517473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:15.517781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:15.519248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:15.537704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 556/1092] 10.222.0.40 () {56 vars in 1133 bytes} [Fri Jul 15 15:14:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:15.543681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:15.634917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:15.656021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4d1d0-e4de-4ff8-8cb8-91f8c76d4808 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:15.690825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c477b4d-46fb-4d64-b3cb-2629873039bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:15.691461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477b4d-46fb-4d64-b3cb-2629873039bf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:15.691743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477b4d-46fb-4d64-b3cb-2629873039bf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:15.692019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c477b4d-46fb-4d64-b3cb-2629873039bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:15.762663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c477b4d-46fb-4d64-b3cb-2629873039bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:15.764161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c477b4d-46fb-4d64-b3cb-2629873039bf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:15.845963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:15.867805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 537/1093] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:15.873854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:15.963595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:15.983552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c84b9-2660-47b7-aaa7-dfbd590b7efc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.019783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.020502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.020777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.021048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.031894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.033356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:16.050983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:16.051988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:16.053062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:14:16.053062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:16.053062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:14:16.053062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:16.053386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67228b-0bb9-4275-80cc-f8859434ede5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.053386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67228b-0bb9-4275-80cc-f8859434ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.053386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67228b-0bb9-4275-80cc-f8859434ede5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.053778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 557/1094] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:16.053872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 538/1095] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:14:16.057570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.058366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.058811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.059094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.077973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:16.079219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:16.152479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 539/1096] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:16.156987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837cc856-0c2c-4d00-b66a-700539e99dc8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.176374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.191239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.191939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.192221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.192498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.193215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:16.193522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.195024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.196493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:16.208663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 558/1097] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:16.214376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.280407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a50a-9d49-495c-aacf-4abfb0400690 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.291086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:16.317327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.318127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.318408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.318714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.319758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ccf294-f284-4065-ba16-aecc1af1b0ba None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.329348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:16.336939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:16.337852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:16.338953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 540/1098] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:14:16.344837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.353552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.354368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.354682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.354962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.362764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.364233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:14:16.378760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:16.379646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:16.380566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 559/1099] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:16.385792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.443156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:16.445162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c19ba-1b17-45c8-99fe-fd837838af5e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.479190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.479853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.480140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.480412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.481161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:16.481471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.482973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.488602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b834798-4041-4e67-a51f-d46b081fbf93 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.491299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 541/1100] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:16.497126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.524109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.524774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.525053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.525323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.526112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:16.526417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.527900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.530877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:16.541509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 560/1101] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:16.548018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.599776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd2243-c537-41b4-8982-5e8163a3dd5f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.634784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.635669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.636216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.636216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] PATH_INFO: `/v3/users/e0b5500218594d83a8ed1929920095f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.637418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=e0b5500218594d83a8ed1929920095f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.638950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.640678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 542/1102] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/users/e0b5500218594d83a8ed1929920095f0 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:16.646669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:16.650075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec494104-f27d-4b02-8ad0-faf2aa2fd884 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.664404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:16.685200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.685940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.686192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.686461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.687409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] RBAC: Adding query filter params (role.id=5c1724e5bd744c02830f4aebfeec8458, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=063c589447364966b64fcc2da5c1aa02) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:16.687710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.689147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.695113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 561/1103] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/role_assignments?user.id=063c589447364966b64fcc2da5c1aa02&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=5c1724e5bd744c02830f4aebfeec8458 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:16.749264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:16.752440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe870dbe-b752-47bd-b568-9651c4af3ab2 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:16.790945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:16.791514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:16.791796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:16.792069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:16.868654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:16.870122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:16.996490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:17.002859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 543/1104] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 15:14:17.007564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:17.112088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c77721-5df9-41c0-83b6-505cb250536f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:17.146899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66be2b-63fa-4aaa-8e77-6d044a868b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:17.147807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66be2b-63fa-4aaa-8e77-6d044a868b6b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:17.148087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66be2b-63fa-4aaa-8e77-6d044a868b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:17.148364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66be2b-63fa-4aaa-8e77-6d044a868b6b None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:17.149532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66be2b-63fa-4aaa-8e77-6d044a868b6b None admin] RBAC: Authorizing `identity:get_project(project_id=bd9a637a1bbc4e629d7196773078e709)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:17.151046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66be2b-63fa-4aaa-8e77-6d044a868b6b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:17.152758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 562/1105] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:17 2022] GET /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:17.161250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:17.269125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eabec5-cd00-4ffd-8d4b-8cc29b633d7d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:17.308537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:17.309200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:17.309478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:17.309787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:17.311563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=bd9a637a1bbc4e629d7196773078e709, user.id=e0b5500218594d83a8ed1929920095f0) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:17.311912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:17.313415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b35da-b99c-4fd8-9ff7-b8119f458627 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:17.320475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 544/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:17 2022] GET /identity/v3/role_assignments?user.id=e0b5500218594d83a8ed1929920095f0&scope.project.id=bd9a637a1bbc4e629d7196773078e709&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:19.365525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ed24e-ffe2-41f8-90c6-14c5e78b59ff None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:19.366091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ed24e-ffe2-41f8-90c6-14c5e78b59ff None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:19.366091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ed24e-ffe2-41f8-90c6-14c5e78b59ff None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:19.367490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 563/1107] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:19.374248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:19.375051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:19.375193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:19.397782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:19.399030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:19.464741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:19.480313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 545/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:19.505418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ed24e-ffe2-41f8-90c6-14c5e78b59ff None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:19.563348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:19.623233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ed24e-ffe2-41f8-90c6-14c5e78b59ff None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:19.658776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab991546-95b1-4e29-91f6-eeeac8ce1e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:19.659587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991546-95b1-4e29-91f6-eeeac8ce1e18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:19.659864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991546-95b1-4e29-91f6-eeeac8ce1e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:19.660132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991546-95b1-4e29-91f6-eeeac8ce1e18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:19.661360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991546-95b1-4e29-91f6-eeeac8ce1e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:19.662874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991546-95b1-4e29-91f6-eeeac8ce1e18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:19.664551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 564/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:19.674001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:19.789189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db004a52-500c-4734-a7a5-9d00257020aa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:19.825947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2711fe-6c6f-4672-8293-4eb2e090214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:19.827202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2711fe-6c6f-4672-8293-4eb2e090214c None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:19.827492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2711fe-6c6f-4672-8293-4eb2e090214c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:19.827773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2711fe-6c6f-4672-8293-4eb2e090214c None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:19.828488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2711fe-6c6f-4672-8293-4eb2e090214c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:19.829944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2711fe-6c6f-4672-8293-4eb2e090214c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:19.851616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 546/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:14:19 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:21.891742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8ba81-fd71-4f16-86bd-2edb41ae1959 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:21.892806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8ba81-fd71-4f16-86bd-2edb41ae1959 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:21.892806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8ba81-fd71-4f16-86bd-2edb41ae1959 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:21.894871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 565/1111] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:21.901295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:21.902055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:21.902212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:21.920929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:21.922123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:21.972595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:21.998343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 547/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:22.021707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8ba81-fd71-4f16-86bd-2edb41ae1959 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:22.133069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8ba81-fd71-4f16-86bd-2edb41ae1959 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:22.147106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:22.168525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:22.169382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:22.169715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:22.170073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] PATH_INFO: `/v3/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:22.171679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] RBAC: Authorizing `identity:get_role(role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:22.173062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:22.174441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 566/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:22 2022] GET /identity/v3/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:22.180691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:22.284821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a3f3e-deb4-4feb-9663-d8fcea3fb846 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:22.320993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:22.321858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:22.322139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:22.322415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] PATH_INFO: `/v3/users/b181a959f84848569524bf799b868aea` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:22.337129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] RBAC: Authorizing `identity:get_user(user_id=b181a959f84848569524bf799b868aea)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:22.338559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:22.340217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 548/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:22 2022] GET /identity/v3/users/b181a959f84848569524bf799b868aea => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:22.349333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:22.455702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac70d9b-b5fa-413a-99fe-5b4e8bf2e53b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:22.492595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:22.493249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:22.493739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:22.494218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:22.548778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:22.567826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:22.569295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:22.671338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 567/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:22.677966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:22.726661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:22.805811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d365b6-265e-47ee-81dd-321bfac8fabb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:22.847084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2c3b78-cd28-4d43-b886-f6d3ae2e2bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:22.847943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c3b78-cd28-4d43-b886-f6d3ae2e2bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:22.848226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c3b78-cd28-4d43-b886-f6d3ae2e2bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:22.848503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c3b78-cd28-4d43-b886-f6d3ae2e2bf4 None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:22.849731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c3b78-cd28-4d43-b886-f6d3ae2e2bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9a637a1bbc4e629d7196773078e709)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:22.851208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2c3b78-cd28-4d43-b886-f6d3ae2e2bf4 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:22.852903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 549/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:22 2022] GET /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:22.863397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:22.952875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:22.973170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb47b13-9709-4ee2-b44b-a9e3cb889818 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:23.008661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:23.009215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:23.009521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:23.009791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:23.010646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] RBAC: Adding query filter params (role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=bd9a637a1bbc4e629d7196773078e709, user.id=b181a959f84848569524bf799b868aea) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:23.010955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:23.012399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c217b9e2-a862-4804-8892-6563fc78c2f7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:23.018250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 568/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:22 2022] GET /identity/v3/role_assignments?user.id=b181a959f84848569524bf799b868aea&scope.project.id=bd9a637a1bbc4e629d7196773078e709&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:24.975504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdafd1f-e474-43e7-a146-de06460959f9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:24.976015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdafd1f-e474-43e7-a146-de06460959f9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:24.976015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdafd1f-e474-43e7-a146-de06460959f9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:24.978155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 550/1118] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:24.987681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:24.988119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:24.988275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:25.010310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:25.011556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:25.082666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 569/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:25.105961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdafd1f-e474-43e7-a146-de06460959f9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:25.239956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdafd1f-e474-43e7-a146-de06460959f9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:25.247358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:25.283843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:25.284706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:25.284990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:25.285263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] PATH_INFO: `/v3/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:25.286750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] RBAC: Authorizing `identity:get_role(role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:25.288130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:25.289499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 551/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:25 2022] GET /identity/v3/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 223 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:25.295703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:25.381446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:25.406563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f51a39-4260-46e5-a5dc-d4ba2d9f74f0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:25.442508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:25.443407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:25.443717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:25.443988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] PATH_INFO: `/v3/users/b181a959f84848569524bf799b868aea` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:25.445302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] RBAC: Authorizing `identity:get_user(user_id=b181a959f84848569524bf799b868aea)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:25.446837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:25.448565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 570/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:25 2022] GET /identity/v3/users/b181a959f84848569524bf799b868aea => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:25.455992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:25.575357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8f32a-9158-4ba0-a433-d39b2be59be6 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:25.609840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a989e-7bbf-49ae-9239-8eb3102885fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:25.611113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a989e-7bbf-49ae-9239-8eb3102885fe None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:25.611393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a989e-7bbf-49ae-9239-8eb3102885fe None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:25.611664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a989e-7bbf-49ae-9239-8eb3102885fe None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:25.612895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a989e-7bbf-49ae-9239-8eb3102885fe None admin] RBAC: Authorizing `identity:get_project(project_id=bd9a637a1bbc4e629d7196773078e709)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:25.614337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a989e-7bbf-49ae-9239-8eb3102885fe None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:25.616079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 552/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:25 2022] GET /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:25.625268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:25.739934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43a7254-3a80-48fc-bd4c-1b2b9533fd6a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:25.777547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f74c5-4014-41ba-b0bd-2a60a32b4a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:25.778989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f74c5-4014-41ba-b0bd-2a60a32b4a19 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:25.779321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f74c5-4014-41ba-b0bd-2a60a32b4a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:25.779620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f74c5-4014-41ba-b0bd-2a60a32b4a19 None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709/users/b181a959f84848569524bf799b868aea/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:25.782496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f74c5-4014-41ba-b0bd-2a60a32b4a19 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9a637a1bbc4e629d7196773078e709, user_id=b181a959f84848569524bf799b868aea, role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:25.783984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f74c5-4014-41ba-b0bd-2a60a32b4a19 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:25.800564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 571/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:14:25 2022] PUT /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709/users/b181a959f84848569524bf799b868aea/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:14:27.815876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa6e0-0b26-4a36-b8e1-259c1354f733 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:27.816786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa6e0-0b26-4a36-b8e1-259c1354f733 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:27.816786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa6e0-0b26-4a36-b8e1-259c1354f733 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:27.818967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 553/1124] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:27.824331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:27.824772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:27.824866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:27.832313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:27.845164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:27.846381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:27.917604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 572/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:27.941822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa6e0-0b26-4a36-b8e1-259c1354f733 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:27.992132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:28.059527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa6e0-0b26-4a36-b8e1-259c1354f733 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:28.097179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:28.097959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:28.098233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:28.098502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] PATH_INFO: `/v3/roles/f54a42a47de04ed48e6a6f46cf86f411` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:28.100156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] RBAC: Authorizing `identity:get_role(role_id=f54a42a47de04ed48e6a6f46cf86f411)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:28.101537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:28.102900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 554/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:27 2022] GET /identity/v3/roles/f54a42a47de04ed48e6a6f46cf86f411 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:28.110276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:28.221951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa34f771-f869-418b-9e90-93bca2f9e7a7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:28.263637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:28.264563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:28.264837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:28.265103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] PATH_INFO: `/v3/users/b181a959f84848569524bf799b868aea` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:28.266420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] RBAC: Authorizing `identity:get_user(user_id=b181a959f84848569524bf799b868aea)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:28.267912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:28.269545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 573/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:28 2022] GET /identity/v3/users/b181a959f84848569524bf799b868aea => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:28.275529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:28.382883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0818f40-8f94-496c-8a64-95d16423e983 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:28.393838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:28.419023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:28.419738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:28.420020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:28.420236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:28.492286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:28.493723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:28.597739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 555/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:28.604425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:28.609700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:28.729041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c0c0-ff42-48e8-b9c3-b59431bd5abe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:28.783101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5fc1b-df8c-4e0c-9087-63dc335f8cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:28.783973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5fc1b-df8c-4e0c-9087-63dc335f8cce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:28.784248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5fc1b-df8c-4e0c-9087-63dc335f8cce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:28.784520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5fc1b-df8c-4e0c-9087-63dc335f8cce None admin] PATH_INFO: `/v3/projects/bd9a637a1bbc4e629d7196773078e709` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:28.785847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5fc1b-df8c-4e0c-9087-63dc335f8cce None admin] RBAC: Authorizing `identity:get_project(project_id=bd9a637a1bbc4e629d7196773078e709)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:28.787340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5fc1b-df8c-4e0c-9087-63dc335f8cce None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:28.789051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 574/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:28 2022] GET /identity/v3/projects/bd9a637a1bbc4e629d7196773078e709 => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:28.795623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:28.807605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:28.911184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff340ff-7f56-4d9e-b406-9b595f6b7765 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:28.948446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:28.949113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:28.949397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:28.949675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:28.950558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] RBAC: Adding query filter params (role.id=f54a42a47de04ed48e6a6f46cf86f411, scope.project.id=bd9a637a1bbc4e629d7196773078e709, user.id=b181a959f84848569524bf799b868aea) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:28.951143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:28.952601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6f247-93b4-4a8f-9fde-f326dba6076b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:28.959178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 556/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:28 2022] GET /identity/v3/role_assignments?user.id=b181a959f84848569524bf799b868aea&scope.project.id=bd9a637a1bbc4e629d7196773078e709&role.id=f54a42a47de04ed48e6a6f46cf86f411 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:31.005359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed125217-e9f9-4f9f-85f0-3ad806c5a749 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:31.006179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed125217-e9f9-4f9f-85f0-3ad806c5a749 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:31.006179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed125217-e9f9-4f9f-85f0-3ad806c5a749 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:31.008530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 575/1131] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:31.016432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:31.016878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:31.016983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:31.035393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:31.036591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:31.106840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 557/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:14:31.130983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed125217-e9f9-4f9f-85f0-3ad806c5a749 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:31.216609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:31.252320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed125217-e9f9-4f9f-85f0-3ad806c5a749 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:31.292019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588e3ba-b54f-4084-a58e-657bc6ebda99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:31.293180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e3ba-b54f-4084-a58e-657bc6ebda99 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:31.293547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e3ba-b54f-4084-a58e-657bc6ebda99 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:31.293901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e3ba-b54f-4084-a58e-657bc6ebda99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:31.295400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588e3ba-b54f-4084-a58e-657bc6ebda99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:31.296875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588e3ba-b54f-4084-a58e-657bc6ebda99 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:31.298550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 576/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:31.304862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:31.308138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:31.408967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ba879f-e6e8-4201-b901-bcaebe462029 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:31.444106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a27dca-a396-433c-b55e-bc6e098220c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:31.444761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27dca-a396-433c-b55e-bc6e098220c9 None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:31.445043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27dca-a396-433c-b55e-bc6e098220c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:31.445318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a27dca-a396-433c-b55e-bc6e098220c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:31.446020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a27dca-a396-433c-b55e-bc6e098220c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:31.447488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a27dca-a396-433c-b55e-bc6e098220c9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:31.482788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 558/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:14:31 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:33.581212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e22f1-f11e-4147-8d92-b9ae1ec29d26 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:33.582072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e22f1-f11e-4147-8d92-b9ae1ec29d26 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:33.582072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e22f1-f11e-4147-8d92-b9ae1ec29d26 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:33.584305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 577/1135] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:33.593170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:33.593869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:33.594148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:33.617610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:33.618831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:33.680760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:33.704096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 559/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:33.738868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e22f1-f11e-4147-8d92-b9ae1ec29d26 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:33.848207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e22f1-f11e-4147-8d92-b9ae1ec29d26 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:33.886254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63ddb8-6e4a-40fc-9b9e-bcd17d4a77e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:33.887084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63ddb8-6e4a-40fc-9b9e-bcd17d4a77e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:33.887383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63ddb8-6e4a-40fc-9b9e-bcd17d4a77e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:33.887642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63ddb8-6e4a-40fc-9b9e-bcd17d4a77e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:33.888998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63ddb8-6e4a-40fc-9b9e-bcd17d4a77e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:33.890458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63ddb8-6e4a-40fc-9b9e-bcd17d4a77e6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:33.892133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 578/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:14:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:33.899101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:34.007851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb9d0e-7460-4ca8-a395-fea50cc5c249 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:34.046843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bbd9d-1552-4718-a026-27b9484cb62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:34.047515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bbd9d-1552-4718-a026-27b9484cb62e None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:34.047800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bbd9d-1552-4718-a026-27b9484cb62e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:34.048073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bbd9d-1552-4718-a026-27b9484cb62e None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:34.048788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bbd9d-1552-4718-a026-27b9484cb62e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:34.050235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bbd9d-1552-4718-a026-27b9484cb62e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:34.077085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 560/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:14:33 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:36.138399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64d79-c6c3-4cfc-aad6-eaf41b75a5ec None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:36.139036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64d79-c6c3-4cfc-aad6-eaf41b75a5ec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:36.139036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff64d79-c6c3-4cfc-aad6-eaf41b75a5ec None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:36.140378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 579/1139] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:36.146824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:36.147579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:36.147724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:36.166575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:36.167778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:36.208585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:36.242213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 561/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:36.266149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64d79-c6c3-4cfc-aad6-eaf41b75a5ec None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:36.276325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:36.400181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff64d79-c6c3-4cfc-aad6-eaf41b75a5ec None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:36.444142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:36.444926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:36.445203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:36.445472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:36.457215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:36.465048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:36.466213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:36.467245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 580/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:36.472955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:36.574012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f67a2-8a48-4094-a1ad-ca34205b5979 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:36.609863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:36.610544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:36.610834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:36.611098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:36.611840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:36.612146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:36.613514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:36.621821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 562/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:36.629110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:36.703218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:36.740508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1257d1-6860-4264-8b9d-a7edc312cedc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:36.774481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:36.775384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:36.775668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:36.775943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] PATH_INFO: `/v3/users/0d05804293bd4424b71d3a03562ea2b0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:36.789259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] RBAC: Authorizing `identity:get_user(user_id=0d05804293bd4424b71d3a03562ea2b0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:36.790762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:36.792425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 581/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:36 2022] GET /identity/v3/users/0d05804293bd4424b71d3a03562ea2b0 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:36.799375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:36.846281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:36.920111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084f56e-5007-4423-b13d-5b9d67462a10 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:36.957598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:36.958253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:36.958563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:36.958909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:37.032259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:37.033635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:37.067635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:37.136593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 563/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:37.146265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:37.256648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e461-e63b-49ce-9309-49ff4947b371 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:37.283103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:37.296343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2e501-e988-4195-861a-98b22ea79b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:37.297180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2e501-e988-4195-861a-98b22ea79b30 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:37.297491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2e501-e988-4195-861a-98b22ea79b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:37.297762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2e501-e988-4195-861a-98b22ea79b30 None admin] PATH_INFO: `/v3/projects/651cd423da9c4d09992bea43eb0cfac1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:37.299128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2e501-e988-4195-861a-98b22ea79b30 None admin] RBAC: Authorizing `identity:get_project(project_id=651cd423da9c4d09992bea43eb0cfac1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:37.300583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2e501-e988-4195-861a-98b22ea79b30 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:37.302306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 582/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:37 2022] GET /identity/v3/projects/651cd423da9c4d09992bea43eb0cfac1 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:37.308912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:37.420497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413b5de-f775-4df0-8688-03c75f17c4da None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:37.455261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:37.455910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:37.456192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:37.456475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:37.457308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=651cd423da9c4d09992bea43eb0cfac1, user.id=0d05804293bd4424b71d3a03562ea2b0) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:37.457617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:37.459084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa3598-5c35-48df-b093-0c860890ec5a None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:37.460936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:37.466840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 564/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:37 2022] GET /identity/v3/role_assignments?user.id=0d05804293bd4424b71d3a03562ea2b0&scope.project.id=651cd423da9c4d09992bea43eb0cfac1&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:39.441520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f810-1f3d-4aae-8114-d15ad075147d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:39.442087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f810-1f3d-4aae-8114-d15ad075147d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:39.442087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f810-1f3d-4aae-8114-d15ad075147d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:39.444298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 583/1147] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:39.451323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:39.451863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:39.451863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:39.478966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:39.480910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:39.555334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 565/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:39.579928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f810-1f3d-4aae-8114-d15ad075147d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:39.704044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f810-1f3d-4aae-8114-d15ad075147d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:39.739503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:39.740301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:39.740585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:39.740857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:39.752202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:39.759749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:39.760732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:39.761732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 584/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:39.767628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:39.871872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877e9d1-dbf5-46cf-b70d-ae0f765f999d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:39.902211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:39.910803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:39.911466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:39.911752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:39.912023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:39.912769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:39.913074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:39.914447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:39.922454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 566/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:39.928599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:39.988125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:40.041989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c516739-706f-43af-8fef-c464d2a5bc31 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:40.081520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:40.082390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:40.082702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:40.082979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] PATH_INFO: `/v3/users/0d05804293bd4424b71d3a03562ea2b0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:40.084292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] RBAC: Authorizing `identity:get_user(user_id=0d05804293bd4424b71d3a03562ea2b0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:40.085744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:40.087446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 585/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:39 2022] GET /identity/v3/users/0d05804293bd4424b71d3a03562ea2b0 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:40.095201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:40.208651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afd2ef-4dcb-4395-afe2-3c2d1c07aefb None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:40.243670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2632a-aa36-4dc4-849f-6ff584fab3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:40.244467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2632a-aa36-4dc4-849f-6ff584fab3bd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:40.244751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2632a-aa36-4dc4-849f-6ff584fab3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:40.245031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2632a-aa36-4dc4-849f-6ff584fab3bd None admin] PATH_INFO: `/v3/projects/651cd423da9c4d09992bea43eb0cfac1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:40.246231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2632a-aa36-4dc4-849f-6ff584fab3bd None admin] RBAC: Authorizing `identity:get_project(project_id=651cd423da9c4d09992bea43eb0cfac1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:40.247724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da2632a-aa36-4dc4-849f-6ff584fab3bd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:40.249441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 567/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:40 2022] GET /identity/v3/projects/651cd423da9c4d09992bea43eb0cfac1 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:40.258872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:40.363963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8116e-62a8-4a9f-9c88-e7a6bdc82971 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:40.383377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:40.398820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480a7bf1-2fd6-49e6-bf69-9db5b6cfbb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:40.399701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a7bf1-2fd6-49e6-bf69-9db5b6cfbb02 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:40.399977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a7bf1-2fd6-49e6-bf69-9db5b6cfbb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:40.400248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a7bf1-2fd6-49e6-bf69-9db5b6cfbb02 None admin] PATH_INFO: `/v3/projects/651cd423da9c4d09992bea43eb0cfac1/users/0d05804293bd4424b71d3a03562ea2b0/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:40.402228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a7bf1-2fd6-49e6-bf69-9db5b6cfbb02 None admin] RBAC: Authorizing `identity:create_grant(project_id=651cd423da9c4d09992bea43eb0cfac1, user_id=0d05804293bd4424b71d3a03562ea2b0, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:40.403915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480a7bf1-2fd6-49e6-bf69-9db5b6cfbb02 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:40.421610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 586/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:14:40 2022] PUT /identity/v3/projects/651cd423da9c4d09992bea43eb0cfac1/users/0d05804293bd4424b71d3a03562ea2b0/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:14:42.420910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef4f71-0bb6-4045-9f35-58b2072d9f5b None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:42.421504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef4f71-0bb6-4045-9f35-58b2072d9f5b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:42.421504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef4f71-0bb6-4045-9f35-58b2072d9f5b None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:42.422889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 568/1154] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:42.429237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:42.429956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:42.430129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:42.451011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:42.452197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:42.523048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 587/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:42.554416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef4f71-0bb6-4045-9f35-58b2072d9f5b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:42.620011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:42.664315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef4f71-0bb6-4045-9f35-58b2072d9f5b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:42.699139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:42.700027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:42.700305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:42.700576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:42.711283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:42.719036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:42.720150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:42.721179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 569/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:42.727116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:42.834143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa629d27-02bb-4984-b4d8-eba6129961e8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:42.845560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:42.876735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:42.877393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:42.877676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:42.877948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:42.878735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:42.879019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:42.880430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:42.889119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 588/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:42.895273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:43.000600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1fe00-6f0d-45a8-9b25-2705fe5ede36 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:43.018954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:43.037107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:43.037987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:43.038221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:43.038512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] PATH_INFO: `/v3/users/0d05804293bd4424b71d3a03562ea2b0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:43.039787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] RBAC: Authorizing `identity:get_user(user_id=0d05804293bd4424b71d3a03562ea2b0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:43.041234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:43.042879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 570/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:42 2022] GET /identity/v3/users/0d05804293bd4424b71d3a03562ea2b0 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:43.049211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:43.159542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f87e8-31ee-47d4-a166-447e317da3ee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:43.196109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:43.196755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:43.197033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:43.197304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:43.246766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:43.275272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:43.276668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:43.378700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 589/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:43.385082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:43.494153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aad5b-41cd-494f-ae5e-006437813c50 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:43.531668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc6cc3-c48f-4fbd-89a1-b307e19a6854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:43.532465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6cc3-c48f-4fbd-89a1-b307e19a6854 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:43.532743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6cc3-c48f-4fbd-89a1-b307e19a6854 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:43.533012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc6cc3-c48f-4fbd-89a1-b307e19a6854 None admin] PATH_INFO: `/v3/projects/651cd423da9c4d09992bea43eb0cfac1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:43.534252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc6cc3-c48f-4fbd-89a1-b307e19a6854 None admin] RBAC: Authorizing `identity:get_project(project_id=651cd423da9c4d09992bea43eb0cfac1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:43.535732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc6cc3-c48f-4fbd-89a1-b307e19a6854 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:43.537468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 571/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:43 2022] GET /identity/v3/projects/651cd423da9c4d09992bea43eb0cfac1 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:43.546421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:43.620715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:43.663987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2d98d-073e-4467-9347-fb94a061e026 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:43.702360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:43.703035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:43.703317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:43.703596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:43.704437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=651cd423da9c4d09992bea43eb0cfac1, user.id=0d05804293bd4424b71d3a03562ea2b0) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:43.704743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:43.706199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b452-95be-4f75-af57-9297b49e7e53 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:43.712288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 590/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:43 2022] GET /identity/v3/role_assignments?user.id=0d05804293bd4424b71d3a03562ea2b0&scope.project.id=651cd423da9c4d09992bea43eb0cfac1&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:45.738749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c8c41-a11a-46ff-984e-880e15811971 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:45.739603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c8c41-a11a-46ff-984e-880e15811971 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:45.739603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c8c41-a11a-46ff-984e-880e15811971 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:45.741749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 572/1162] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:45.750621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:45.751473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:45.751658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:45.774444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:45.775671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:45.848757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 591/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:45.871304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c8c41-a11a-46ff-984e-880e15811971 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:45.927642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:45.980242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c8c41-a11a-46ff-984e-880e15811971 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:46.018827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:46.019410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:46.019699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:46.019975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:46.094729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:46.096167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:46.151277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:46.202127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 573/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:46.211001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:46.282932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:46.323105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1e31-5cc5-42c4-94b5-6135553a3e28 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:46.361820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:46.362649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:46.362931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:46.363209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:46.369978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:46.371496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2251f107-2db6-4d84-ba51-8e6f786aae44 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:46.386153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:46.387519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:14:46.388425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 592/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:14:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:46.393987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:46.502204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ab847-25da-40ba-ab2e-ec89f1662553 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:46.540554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:46.541383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:46.541766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:46.542145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:46.542943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:46.543315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:46.545063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2884c1d-2bbb-4469-b4c9-5493a5f4b823 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:46.552588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 574/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Fri Jul 15 15:14:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:14:48.545151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296304bc-0e1b-4c70-905c-37679bd12fe2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:48.545715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296304bc-0e1b-4c70-905c-37679bd12fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:48.545715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296304bc-0e1b-4c70-905c-37679bd12fe2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:48.547161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 593/1167] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:14:48.553545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaac422-a750-4892-b4d9-fdee0f1770af None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:48.554229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaac422-a750-4892-b4d9-fdee0f1770af None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:48.554425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaac422-a750-4892-b4d9-fdee0f1770af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:48.580410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0aaac422-a750-4892-b4d9-fdee0f1770af None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:48.581612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0aaac422-a750-4892-b4d9-fdee0f1770af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:48.595345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:48.657560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 575/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:48.688391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296304bc-0e1b-4c70-905c-37679bd12fe2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:48.801164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296304bc-0e1b-4c70-905c-37679bd12fe2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:48.827305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:48.840208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956bc79-5098-4a7e-bbf0-7a6bd8400c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:48.840872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956bc79-5098-4a7e-bbf0-7a6bd8400c8d None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:48.841153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956bc79-5098-4a7e-bbf0-7a6bd8400c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:48.841424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956bc79-5098-4a7e-bbf0-7a6bd8400c8d None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:48.842170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956bc79-5098-4a7e-bbf0-7a6bd8400c8d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:48.843594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956bc79-5098-4a7e-bbf0-7a6bd8400c8d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:48.863018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 594/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Fri Jul 15 15:14:48 2022] POST /identity/v3/domains => generated 267 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:50.880750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc677e-c1f8-4ae0-8cca-ecff23de423a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:50.881343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc677e-c1f8-4ae0-8cca-ecff23de423a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:50.881343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc677e-c1f8-4ae0-8cca-ecff23de423a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:50.882714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 576/1170] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:50.888977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:50.889725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:50.889826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:50.918362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:50.919587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:50.994135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 595/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:51.018153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc677e-c1f8-4ae0-8cca-ecff23de423a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:51.133696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc677e-c1f8-4ae0-8cca-ecff23de423a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:51.168082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4516bd5-bfb3-4984-8e7c-c411a4cbe0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:51.168931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516bd5-bfb3-4984-8e7c-c411a4cbe0ea None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:51.169206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516bd5-bfb3-4984-8e7c-c411a4cbe0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:51.169478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516bd5-bfb3-4984-8e7c-c411a4cbe0ea None admin] PATH_INFO: `/v3/domains/b6f871b9726c4f35829a3a8f2cf06b0d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:51.180003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4516bd5-bfb3-4984-8e7c-c411a4cbe0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=b6f871b9726c4f35829a3a8f2cf06b0d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:51.181424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4516bd5-bfb3-4984-8e7c-c411a4cbe0ea None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:51.183219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 577/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Jul 15 15:14:51 2022] GET /identity/v3/domains/b6f871b9726c4f35829a3a8f2cf06b0d => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:51.189589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:51.290815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030f590-2f09-43a7-be56-32c79746da65 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:51.307267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:51.327649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6a2c9-8915-4e32-bf83-f957275d53d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:51.328296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a2c9-8915-4e32-bf83-f957275d53d2 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:51.328569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a2c9-8915-4e32-bf83-f957275d53d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:51.328835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6a2c9-8915-4e32-bf83-f957275d53d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:51.329539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6a2c9-8915-4e32-bf83-f957275d53d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:51.331013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6a2c9-8915-4e32-bf83-f957275d53d2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:51.357879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 596/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:14:51 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:14:53.388504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de7a68-cb54-4faf-9879-37c317c964c8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:53.389452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de7a68-cb54-4faf-9879-37c317c964c8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:53.389452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de7a68-cb54-4faf-9879-37c317c964c8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:53.391606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 578/1174] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:53.397911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:53.398677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:53.398992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:53.417707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:53.418944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:53.489982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 597/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:53.516767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de7a68-cb54-4faf-9879-37c317c964c8 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:53.528790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:53.634428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de7a68-cb54-4faf-9879-37c317c964c8 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:53.672129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b00f3-cc30-40cf-844c-02f8625143f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:53.672924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b00f3-cc30-40cf-844c-02f8625143f8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:53.673206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b00f3-cc30-40cf-844c-02f8625143f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:53.673479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b00f3-cc30-40cf-844c-02f8625143f8 None admin] PATH_INFO: `/v3/domains/b6f871b9726c4f35829a3a8f2cf06b0d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:53.674744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b00f3-cc30-40cf-844c-02f8625143f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b6f871b9726c4f35829a3a8f2cf06b0d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:53.676177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b00f3-cc30-40cf-844c-02f8625143f8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:53.677787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 579/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Fri Jul 15 15:14:53 2022] GET /identity/v3/domains/b6f871b9726c4f35829a3a8f2cf06b0d => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:53.684042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:53.784064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:53.787888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575d5a0-0d54-4d97-b307-50d84b8e4b3a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:53.822466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e84e2-4909-4159-9ffb-ab384f46895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:53.823222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e84e2-4909-4159-9ffb-ab384f46895f None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:53.823503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e84e2-4909-4159-9ffb-ab384f46895f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:53.823821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e84e2-4909-4159-9ffb-ab384f46895f None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:53.824811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e84e2-4909-4159-9ffb-ab384f46895f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:53.826273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e84e2-4909-4159-9ffb-ab384f46895f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:53.847780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 598/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:14:53 2022] POST /identity/v3/users => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:14:55.952253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cea834-b5ae-4722-99cb-2d3d1b8a0d4e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:55.952721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cea834-b5ae-4722-99cb-2d3d1b8a0d4e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:55.952810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cea834-b5ae-4722-99cb-2d3d1b8a0d4e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:55.954124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 580/1178] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:55.959755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:55.960195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:55.960305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:55.978569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:55.979778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:56.051810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 599/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 15:14:56.084394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cea834-b5ae-4722-99cb-2d3d1b8a0d4e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:56.177786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:56.204896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cea834-b5ae-4722-99cb-2d3d1b8a0d4e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:56.239466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:56.240272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:56.240550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:56.240820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:56.251177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:56.258729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:56.259881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:56.261416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 581/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:56.269173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:56.362583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:56.380990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c46f9-941d-4975-942a-d4480c297b29 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:56.420714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:56.421378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:56.421657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:56.421938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:56.422718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:56.423029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:56.424414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:56.433157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 600/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:56.439204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:56.552479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f40310-b187-45f2-86e5-eceea4f13815 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:56.593406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:56.594274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:56.594565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:56.594860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] PATH_INFO: `/v3/users/9d77871b0e3848bcacac60cad95b5c73` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:56.598107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:56.610127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] RBAC: Authorizing `identity:get_user(user_id=9d77871b0e3848bcacac60cad95b5c73)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:56.611613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:56.613269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 582/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:56 2022] GET /identity/v3/users/9d77871b0e3848bcacac60cad95b5c73 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:56.619982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:56.730542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c7f4bb-6245-4334-ae3e-996c89a5cf7f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:56.767084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:56.767726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:56.768002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:56.768276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:56.784160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:56.843273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:56.844687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:56.955972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 601/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:14:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:14:56.965243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:57.068325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acee21-0ea0-4f4e-b128-ca77129b991d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:57.102917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5f63d-2271-4c7c-8f6f-a54259c833bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:57.103701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f63d-2271-4c7c-8f6f-a54259c833bf None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:57.103998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f63d-2271-4c7c-8f6f-a54259c833bf None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:57.104286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5f63d-2271-4c7c-8f6f-a54259c833bf None admin] PATH_INFO: `/v3/projects/80ee24569e2040a595584c56befdaa1e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:57.105537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5f63d-2271-4c7c-8f6f-a54259c833bf None admin] RBAC: Authorizing `identity:get_project(project_id=80ee24569e2040a595584c56befdaa1e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:57.106996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5f63d-2271-4c7c-8f6f-a54259c833bf None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:57.108997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 583/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:56 2022] GET /identity/v3/projects/80ee24569e2040a595584c56befdaa1e => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:57.115423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:57.162356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:57.220671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caaa22-671b-4207-80d2-22c23dc140a8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:57.256201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:57.256853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:57.257127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:57.257399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:57.258228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=80ee24569e2040a595584c56befdaa1e, user.id=9d77871b0e3848bcacac60cad95b5c73) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:57.258570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:57.260001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358d708-65c4-4e84-905d-b4474fa49710 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:57.266177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 602/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:14:57 2022] GET /identity/v3/role_assignments?user.id=9d77871b0e3848bcacac60cad95b5c73&scope.project.id=80ee24569e2040a595584c56befdaa1e&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:59.219284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0190b-79f3-4203-8d27-c0b169dfb471 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:59.219845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0190b-79f3-4203-8d27-c0b169dfb471 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:59.219845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0190b-79f3-4203-8d27-c0b169dfb471 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:59.221209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 584/1186] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:14:59.227735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:59.228472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:59.228626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:59.249038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:14:59.250230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:14:59.321125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 603/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:14:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:14:59.350835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0190b-79f3-4203-8d27-c0b169dfb471 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:59.419325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:59.459929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0190b-79f3-4203-8d27-c0b169dfb471 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:59.495739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:59.496439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:59.496719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:59.496994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:59.509064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:14:59.516705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:14:59.517776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:14:59.518848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 585/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:14:59.524938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:59.571652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:59.634509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb036f5f-2e5b-4e93-803b-25c9baea9836 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:59.671248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:59.671904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:59.672188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:59.672463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:59.673216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:14:59.673521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:59.674954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:59.683435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 604/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:14:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:59.689461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:59.794492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909315e-1cde-46b3-bbbc-62a9f1912f7b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:59.822898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:14:59.842728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:14:59.843900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:14:59.844184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:14:59.844530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] PATH_INFO: `/v3/users/9d77871b0e3848bcacac60cad95b5c73` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:14:59.846655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] RBAC: Authorizing `identity:get_user(user_id=9d77871b0e3848bcacac60cad95b5c73)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:14:59.848965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:14:59.851650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 586/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:14:59 2022] GET /identity/v3/users/9d77871b0e3848bcacac60cad95b5c73 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:14:59.859313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:14:59.971740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1b1b7-e486-4383-aad7-100a42cdb135 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:14:59.973323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:00.009186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1d359-d7e2-4a54-9496-56c691bf78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:00.010079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1d359-d7e2-4a54-9496-56c691bf78e0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:00.010354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1d359-d7e2-4a54-9496-56c691bf78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:00.010661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1d359-d7e2-4a54-9496-56c691bf78e0 None admin] PATH_INFO: `/v3/projects/80ee24569e2040a595584c56befdaa1e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:00.012008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1d359-d7e2-4a54-9496-56c691bf78e0 None admin] RBAC: Authorizing `identity:get_project(project_id=80ee24569e2040a595584c56befdaa1e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:00.013452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1d359-d7e2-4a54-9496-56c691bf78e0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:00.015173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 605/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:14:59 2022] GET /identity/v3/projects/80ee24569e2040a595584c56befdaa1e => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:15:00.024157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:00.129522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5632ea3-92da-45cd-8471-7a156e9682db None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:00.167128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f597bc0-1632-4a3c-aee2-28cd366c71b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:00.168007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f597bc0-1632-4a3c-aee2-28cd366c71b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:00.168277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f597bc0-1632-4a3c-aee2-28cd366c71b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:00.168541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f597bc0-1632-4a3c-aee2-28cd366c71b2 None admin] PATH_INFO: `/v3/projects/80ee24569e2040a595584c56befdaa1e/users/9d77871b0e3848bcacac60cad95b5c73/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:00.170567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f597bc0-1632-4a3c-aee2-28cd366c71b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=80ee24569e2040a595584c56befdaa1e, user_id=9d77871b0e3848bcacac60cad95b5c73, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:00.172165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f597bc0-1632-4a3c-aee2-28cd366c71b2 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:00.188323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 587/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:15:00 2022] PUT /identity/v3/projects/80ee24569e2040a595584c56befdaa1e/users/9d77871b0e3848bcacac60cad95b5c73/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:15:02.180263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66decef-9756-4b58-bbf7-c2fdd322bcf6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:02.180818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66decef-9756-4b58-bbf7-c2fdd322bcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:02.180818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66decef-9756-4b58-bbf7-c2fdd322bcf6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:02.182209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 606/1193] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:15:02.187792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:02.188291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:02.188410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:02.210323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:15:02.211583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:15:02.264505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:02.298963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 588/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:15:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:15:02.330302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66decef-9756-4b58-bbf7-c2fdd322bcf6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:02.436258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66decef-9756-4b58-bbf7-c2fdd322bcf6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:02.473296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:02.474004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:02.474280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:02.474555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:02.486240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:15:02.493835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:15:02.494886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:15:02.496018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 607/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:15:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:15:02.503079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:02.616273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aca68e-a595-42cd-bee3-b35eadc7e6fa None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:02.652085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:02.652778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:02.653025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:02.653305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:02.654057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:15:02.654371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:02.655849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:02.663978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 589/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:15:02.670405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:02.702590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:02.780132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1352aa-8c05-4e89-a61a-6425b84212e4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:02.819905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:02.820783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:02.821063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:02.821426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] PATH_INFO: `/v3/users/9d77871b0e3848bcacac60cad95b5c73` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:02.822730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] RBAC: Authorizing `identity:get_user(user_id=9d77871b0e3848bcacac60cad95b5c73)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:02.824532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:02.826478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 608/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Fri Jul 15 15:15:02 2022] GET /identity/v3/users/9d77871b0e3848bcacac60cad95b5c73 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:15:02.832623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:02.883448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:02.955389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c1836-9c99-4109-bd30-2598b27dbc8e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:02.993453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:02.994119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:02.994401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:02.994714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:03.076541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:03.077913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:03.133563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:03.204038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 590/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:15:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:15:03.206464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:03.307297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:03.311701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1722d-1d21-4003-93fd-944b60f7f249 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:03.347785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e522ee-4ddc-4cd1-8440-7a66dd7e6c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:03.348560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e522ee-4ddc-4cd1-8440-7a66dd7e6c71 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:03.348838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e522ee-4ddc-4cd1-8440-7a66dd7e6c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:03.349111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e522ee-4ddc-4cd1-8440-7a66dd7e6c71 None admin] PATH_INFO: `/v3/projects/80ee24569e2040a595584c56befdaa1e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:03.350409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e522ee-4ddc-4cd1-8440-7a66dd7e6c71 None admin] RBAC: Authorizing `identity:get_project(project_id=80ee24569e2040a595584c56befdaa1e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:03.351891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e522ee-4ddc-4cd1-8440-7a66dd7e6c71 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:03.353597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 609/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Fri Jul 15 15:15:03 2022] GET /identity/v3/projects/80ee24569e2040a595584c56befdaa1e => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:15:03.360722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:03.473571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091e1f8-a3f7-4fce-bee3-844d91225777 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:03.509312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:03.509961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:03.510243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:03.510623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:03.511548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=80ee24569e2040a595584c56befdaa1e, user.id=9d77871b0e3848bcacac60cad95b5c73) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:15:03.511868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:03.513296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a31cd-7e5f-4fb5-bacf-3f3df27ff2b9 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:03.519213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 591/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:15:03 2022] GET /identity/v3/role_assignments?user.id=9d77871b0e3848bcacac60cad95b5c73&scope.project.id=80ee24569e2040a595584c56befdaa1e&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:15:13.342454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=17 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:13.516380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:59.231777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8cbb4-18d0-4770-a207-60cd7bbaba31 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:59.232599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8cbb4-18d0-4770-a207-60cd7bbaba31 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:59.232599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8cbb4-18d0-4770-a207-60cd7bbaba31 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:59.233604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 610/1201] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:15:59.239246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:59.239749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:59.239846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:59.260834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:15:59.262029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:15:59.342715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 592/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:15:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:15:59.364484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8cbb4-18d0-4770-a207-60cd7bbaba31 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:59.490194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8cbb4-18d0-4770-a207-60cd7bbaba31 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:59.547561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:59.548641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:59.548847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:59.549229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:59.562560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:59.564422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:15:59.593608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:15:59.594881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:15:59.595907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:15:59.595907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:15:59.595907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:15:59.595907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:15:59.595907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 611/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:15:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:15:59.604023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:59.738758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e992e-0da8-41ea-bfbb-b3f61682f8f4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:59.772641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:59.783486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:59.784163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:59.784466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:59.784712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:59.785570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:15:59.785884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:59.787296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea03339-e9d9-4ea4-9089-1da8b3e4269e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:15:59.802124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 593/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:15:59.808619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:15:59.883482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:15:59.949074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab786e3-c0c3-4ad0-8322-5b9d48da065f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:15:59.988927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a213a67-c16d-47d1-89de-8260d9c7df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:15:59.989728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a213a67-c16d-47d1-89de-8260d9c7df17 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:15:59.990008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a213a67-c16d-47d1-89de-8260d9c7df17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:15:59.990282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a213a67-c16d-47d1-89de-8260d9c7df17 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:15:59.991040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a213a67-c16d-47d1-89de-8260d9c7df17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:15:59.992778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a213a67-c16d-47d1-89de-8260d9c7df17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:00.026641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 612/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Fri Jul 15 15:15:59 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:16:02.426984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecb578-bc4c-4bd7-8ecf-276d767b174c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:02.426984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecb578-bc4c-4bd7-8ecf-276d767b174c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:02.428076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecb578-bc4c-4bd7-8ecf-276d767b174c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:02.429678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 594/1206] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:02.438944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:02.438944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:02.438944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:02.456835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:02.458435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:02.542993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 613/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:02.567946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecb578-bc4c-4bd7-8ecf-276d767b174c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:02.697781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecb578-bc4c-4bd7-8ecf-276d767b174c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:02.744602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:02.745526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:02.745858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:02.746253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:02.758475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:16:02.767347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:02.768533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:16:02.770047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:02.774591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 595/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:16:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:02.775058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:02.824650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:02.895642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe547df-ab1f-4bd7-a2b5-7cdce86b2675 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:02.936429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:02.939346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:02.939778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:02.940804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:02.941170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:02.941489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:02.943011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:02.955730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 614/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:16:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:02.962095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:03.027837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:03.094827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba9b50-fa57-4cbb-8421-f5ff91c46e5b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:03.137419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:03.138238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:03.138489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:03.138787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:03.150506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:03.151981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:03.169618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:03.170872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:03.171783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:03.171783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:03.171783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:03.171783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:03.171783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 596/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:03.177284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:03.296606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:03.300412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee198b-32a3-4369-83ec-5ca3facad904 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:03.356822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:03.357698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:03.358103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:03.358453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:03.363093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:03.363465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:03.365237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:03.385618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 615/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:03.387016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:03.514860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4ed02-d1c4-4a45-ab0e-a21e898e3869 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:03.517231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:03.558112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:03.558112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:03.558112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:03.558556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:03.568020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:03.570110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:16:03.589558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:03.590702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:03.591696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 597/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:03.598893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:03.738324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326683a1-8732-4140-9422-ac27e497f76a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:03.785866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:03.786651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:03.787017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:03.787282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:03.788292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:03.788725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:03.790371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:03.817795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 616/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Jul 15 15:16:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:16:03.824248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:03.958256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d544f-b714-4b45-ad6b-579c4b8370b7 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:04.003064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:04.003765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:04.004059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:04.004349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:04.048343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:04.099593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:04.101109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:04.234988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 598/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:16:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:16:04.242653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:04.265505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:04.376300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66776aa1-018a-4c8b-b6ff-8ee9df5fb81b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:04.425065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:04.426006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:04.426346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:04.426677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:04.441596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:04.444443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:04.473002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:04.474287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:04.481366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:04.481366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:04.481366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:04.481366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:04.481366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 617/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:04.481672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:04.560201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:04.612593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ea98f-189e-4b31-92f8-243133e80d8c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:04.664879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:04.665669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:04.666028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:04.666370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:04.667164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:04.667516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:04.669190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:04.685715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 599/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:04.691893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:04.777976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:04.817103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d069af45-d4a8-4963-b6ec-0cf73fd91c3d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:04.864038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:04.865605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:04.865605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:04.865795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:04.872819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:04.874286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:16:04.892007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:04.895006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:04.896152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 618/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:04.902797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:05.032762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2188064-1c20-48d8-a05d-b7e379570cd1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:05.066904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:05.095359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50294525-512c-4424-82c3-cff364955444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:05.096046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50294525-512c-4424-82c3-cff364955444 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:05.096433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50294525-512c-4424-82c3-cff364955444 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:05.096725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50294525-512c-4424-82c3-cff364955444 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:05.097683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50294525-512c-4424-82c3-cff364955444 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:05.098045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50294525-512c-4424-82c3-cff364955444 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:05.099753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50294525-512c-4424-82c3-cff364955444 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:05.114275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 600/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:05.120680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:05.228729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57561a0-c68c-42ee-a2c6-9b070e49a4f3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:05.265684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:05.272368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:05.273052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:05.273330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:05.273599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:05.274432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=13549f6dc97241f28501b4c3e3c25959) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:05.274764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:05.276220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58e7c4a-cbe5-4ba2-b3dd-3fc8c16339b7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:05.290563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 619/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:16:05 2022] GET /identity/v3/role_assignments?user.id=13549f6dc97241f28501b4c3e3c25959&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:07.316766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ba0f8-1cb4-4584-b7ee-78a45bf9ecee None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:07.317370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ba0f8-1cb4-4584-b7ee-78a45bf9ecee None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:07.317370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ba0f8-1cb4-4584-b7ee-78a45bf9ecee None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:07.318775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 601/1220] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:07.324190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:07.325624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:07.325624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:07.342168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:07.343296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:07.422862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 620/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:07.445267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ba0f8-1cb4-4584-b7ee-78a45bf9ecee None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:07.569968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ba0f8-1cb4-4584-b7ee-78a45bf9ecee None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:07.608106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:07.608905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:07.609180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:07.609451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:07.621187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:16:07.628827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:07.629875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:07.630767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 602/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:16:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:07.636913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:07.750207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e091d4-a715-4785-af84-0184786b6e60 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:07.775135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:07.787232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:07.787915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:07.788229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:07.788605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:07.789350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:07.789650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:07.791090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:07.804398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 621/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:16:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:07.805873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:07.852219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:07.924524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929a5a-deb9-4ec6-850f-90fe1da3d250 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:07.968818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:07.969755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:07.970085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:07.970492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:07.983672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:07.985559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:08.006214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:08.007937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:08.008951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:08.008951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:08.008951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:08.008951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:08.008951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 603/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:08.013879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:08.126137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f311878-32ed-4db7-be2f-ea80db49a5a1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:08.163052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:08.164208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:08.164532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:08.164810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:08.165625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:08.165988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:08.167445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:08.180995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 622/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:08.187186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:08.311578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:08.319079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69790147-77df-4e3d-b099-2abac43b7a59 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:08.356029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:08.356914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:08.357211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:08.357503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:08.366196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:08.368023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:16:08.385023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:08.386386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:08.387692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 604/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:08.392855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:08.446461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:08.504094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d8e90-9b1e-48e5-af5c-7b8b11b747ed None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:08.539499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:08.540571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:08.540876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:08.541134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:08.541983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:08.542294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:08.543790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:08.561719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 623/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Jul 15 15:16:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:08.567797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:08.691119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6478d4-a37c-4520-a9f1-7962199e2455 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:08.731640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:08.732443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:08.732725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:08.732997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:08.743866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:08.745343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:08.763250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:08.764377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:08.765328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:08.765328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:08.765328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:08.765328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:08.765328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 605/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:08.770675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:08.896849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10079ba2-3267-449c-8578-7d97515af0f1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:08.900328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:08.943187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:08.943880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:08.944260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:08.944540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:08.945392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:08.945702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:08.947219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:08.960668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 624/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:08.966344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:08.979431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:09.098062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d4959-270b-4367-a61a-8709eee0b438 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:09.135197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:09.136094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:09.136383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:09.136698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:09.144124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:09.145573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:16:09.162105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:09.163400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:09.164478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 606/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:16:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:09.170134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:09.279037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2055cd-625b-4943-b6b6-2458b647a13e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:09.322047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:09.322789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:09.323075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:09.323350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:09.324251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:09.324598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:09.326156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edacde03-de04-40b2-bd10-b93d9fe96451 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:09.340843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 625/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:16:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:09.347553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:09.421003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:09.459471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddf86a-728e-4ed2-a6f5-d6f9876f2cce None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:09.497603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ec2fb-e411-4181-8509-0c7f67257bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:09.498581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec2fb-e411-4181-8509-0c7f67257bd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:09.498876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec2fb-e411-4181-8509-0c7f67257bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:09.499178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ec2fb-e411-4181-8509-0c7f67257bd1 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/13549f6dc97241f28501b4c3e3c25959/roles/a13bb79606414b90ae2d6efefde7be5a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:09.514901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ec2fb-e411-4181-8509-0c7f67257bd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=13549f6dc97241f28501b4c3e3c25959, role_id=a13bb79606414b90ae2d6efefde7be5a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:09.516355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ec2fb-e411-4181-8509-0c7f67257bd1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:09.529649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 607/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:16:09 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/13549f6dc97241f28501b4c3e3c25959/roles/a13bb79606414b90ae2d6efefde7be5a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:16:11.653998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69da8e5-a7f9-4228-8f70-e6175eecd9a8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:11.654674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69da8e5-a7f9-4228-8f70-e6175eecd9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:11.654674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69da8e5-a7f9-4228-8f70-e6175eecd9a8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:11.658567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 626/1233] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:11.662033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:11.662619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:11.662738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:11.683279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:11.684635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:11.759458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 608/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:11.789510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69da8e5-a7f9-4228-8f70-e6175eecd9a8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:11.818256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:11.906223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69da8e5-a7f9-4228-8f70-e6175eecd9a8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:11.952659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:11.953495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:11.953776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:11.954045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:11.965060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:16:11.972785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:11.973869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:11.974901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 627/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Fri Jul 15 15:16:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:11.981255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:12.030555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:12.087387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2ed21-f413-476e-9471-3379da24aab1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:12.123630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:12.124306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:12.124572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:12.124847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:12.125600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] RBAC: Adding query filter params (name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:12.125908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:12.127520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:12.135883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 609/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Fri Jul 15 15:16:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:12.141892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:12.237315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:12.255875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f3f6f-5282-47b1-bf55-23f0e6da5bd7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:12.293430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:12.294268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:12.294577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:12.294860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:12.307063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:12.308744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:12.326905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:12.328598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:12.329961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:12.329961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:12.329961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:12.329961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:12.329961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 628/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:12.334952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:12.441799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d878-5742-439b-9947-fb6940cf7088 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:12.451432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:12.476818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:12.477852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:12.478136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:12.478409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:12.479173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:12.479478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:12.480853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:12.493420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 610/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:12.499317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:12.621340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a33343-6747-4b48-b008-0e7438e37a92 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:12.659228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:12.660100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:12.660437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:12.660810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:12.665426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:12.682970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:12.684518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:16:12.707427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:12.708735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:12.710392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:12.711123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 629/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:12.716569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:12.846401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c98f8-1012-4044-bab0-f6a91763a48c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:12.886584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:12.887297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:12.887599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:12.887890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:12.888803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:12.889160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:12.890707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:12.915472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 611/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Jul 15 15:16:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:12.916841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:13.036694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e6243-4412-4d0d-ac40-c628ed5c9b67 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:13.083062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:13.083791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:13.084130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:13.084519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:13.153610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:13.160268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:13.161754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:13.270212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 630/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:16:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:16:13.276829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:13.309971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:13.395398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a9267-85e4-496b-bb08-c529df8fc59e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:13.434487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:13.435405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:13.435733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:13.436010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:13.447559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:13.449085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:13.468036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:13.469130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:13.470163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:13.470163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:13.470163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:13.470163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:13.470163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 612/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:13.478547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:13.577692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:13.584122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b06300-4e69-460c-b214-e4ecdc0ea00e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:13.627535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:13.628199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:13.628586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:13.628746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:13.629652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:13.630030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:13.632510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:13.651570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 631/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:16:13.652948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:13.747775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:13.773772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f7631-1ee7-444f-9b0a-dbfb4459d7e0 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:13.811484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:13.812644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:13.813014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:13.813383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:13.821138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:13.822592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:16:13.839280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:13.840257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:13.841302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 613/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:13.847307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:13.968584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d42e45-5552-4edf-a5e1-49ef2f4a2bb3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:14.010661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:14.011312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:14.011555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:14.011985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:14.014053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:14.014053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:14.014938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f221e35-cc00-42ed-8748-b5b717a7f5bd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:14.028769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 632/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:14.036128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:14.142813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50b6d9-71f8-4451-876e-73bc6c36828c None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:14.177148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:14.178022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:14.178797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:14.178797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:14.179855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] RBAC: Adding query filter params (role.id=a13bb79606414b90ae2d6efefde7be5a, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=13549f6dc97241f28501b4c3e3c25959) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:14.180296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:14.182022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340a075-f509-4eb3-b06b-1512d2f4b128 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:14.189186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 614/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:16:14 2022] GET /identity/v3/role_assignments?user.id=13549f6dc97241f28501b4c3e3c25959&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=a13bb79606414b90ae2d6efefde7be5a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:16.236590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b614-06ed-4af9-af7a-adec7ca65086 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:16.237147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b614-06ed-4af9-af7a-adec7ca65086 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:16.237147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b614-06ed-4af9-af7a-adec7ca65086 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:16.238832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 633/1247] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:16.244402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:16.244840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:16.244953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:16.250275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:16.266180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:16.268712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:16.366794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 615/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:16.394752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1b614-06ed-4af9-af7a-adec7ca65086 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:16.418573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:16.514538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1b614-06ed-4af9-af7a-adec7ca65086 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:16.558006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:16.558827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:16.559107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:16.559392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:16.570804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:16:16.579233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:16.580400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:16.581558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 634/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:16:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:16.587452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:16.704061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d9ac9-a81f-4db8-b71b-e1361d074c18 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:16.737163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:16.742890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:16.743596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:16.743879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:16.744151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:16.744912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:16.745212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:16.746690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:16.756059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 616/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:16:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:16.764333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:16.854878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:16.881298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a0b36-6e4b-4d8a-9feb-9a60536863b6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:16.917607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:16.918443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:16.918751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:16.919033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:16.933441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:16.935952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:16.963694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:16.964876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:16.966004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:16.966004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:16.966004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:16.966004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:16.966377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 635/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:16.973436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:17.086774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb932-df06-4a6b-ac8c-65117b315947 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:17.127624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:17.128375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:17.128699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:17.128975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:17.129775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:17.130087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:17.131841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:17.145028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 617/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:17.151794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:17.262158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76c51f-7cae-456e-bfa6-31ea64ad2c8a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:17.293250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:17.302856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:17.303865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:17.304187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:17.304567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:17.313276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:17.315035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:16:17.331350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:17.332550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:17.333784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:17.338764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 636/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:17.340024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:17.404169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:17.458401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde73b15-a7a4-414e-b40b-24a61c45f5f8 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:17.496836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:17.497666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:17.498047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:17.498438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:17.499448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:17.499754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:17.501230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:17.524652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 618/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Jul 15 15:16:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:17.531175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:17.651611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220fb60-8454-4d3b-9517-9e9e20494934 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:17.699443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:17.700155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:17.700443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:17.700790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:17.770593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:17.784990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:17.786825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:17.907029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 637/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:16:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:16:17.913817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:18.027955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe186b-b808-4191-b5c1-fa4ca00b9da1 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:18.045009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:18.070067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:18.070932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:18.071190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:18.071466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:18.084609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:18.086101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:18.106396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:18.107696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:18.108759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:18.108759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:18.108759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:18.108759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:18.108759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 619/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:18.115064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:18.203716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:18.217923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32a402-64b0-491f-aa36-d876dd2e3715 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:18.253194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:18.253868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:18.254166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:18.254414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:18.255179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:18.255486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:18.257076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:18.269534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 638/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:16:18.275692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:18.397229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffdfd7-4321-4609-ba7e-cf69e25702a3 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:18.441486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:18.442530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:18.442903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:18.443254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:18.451571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:18.453290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:16:18.469207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:18.470289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:18.471365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 620/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:18.478107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:18.588685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73c013-1498-4e28-8553-9435d25e4d79 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:18.627829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:18.628776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:18.629192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:18.629617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:18.630873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:18.631796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:18.633267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a03e76-0cce-433b-9e12-ee349095e413 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:18.645564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 639/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:16:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:16:18.652508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:18.684039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:18.773289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f1e45-7621-4c76-afee-99d033f41d67 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:18.813532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:18.814267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:18.814539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:18.814830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:18.815665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=13549f6dc97241f28501b4c3e3c25959) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:18.815974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:18.817407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1719f3-e29d-49f4-b6de-41415525bdde None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:18.825141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 621/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:16:18 2022] GET /identity/v3/role_assignments?user.id=13549f6dc97241f28501b4c3e3c25959&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:20.917314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07141de0-579c-4d86-9e91-de9a6a85630c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:20.917870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07141de0-579c-4d86-9e91-de9a6a85630c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:20.917870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07141de0-579c-4d86-9e91-de9a6a85630c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:20.920083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 640/1261] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:20.929177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:20.929599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:20.929726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:20.953221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:20.954462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:21.025318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 622/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:21.049353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07141de0-579c-4d86-9e91-de9a6a85630c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:21.062930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:21.183004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07141de0-579c-4d86-9e91-de9a6a85630c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:21.216593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:21.217387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:21.217675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:21.217949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:21.228281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:16:21.235856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:21.236948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:21.244675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 641/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:16:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:21.245558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:21.264644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:21.353655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c91fa8-2e44-449e-b7dc-5ed58198ad10 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:21.394282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:21.395056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:21.395336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:21.395608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:21.396499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:21.396896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:21.398187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:21.406384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 623/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:16:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:21.412523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:21.479350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:21.516069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f28c2-0e41-4445-805b-bcdf3dfe4706 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:21.550960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:21.551795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:21.552071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:21.552341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:21.563178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:21.564789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:21.581984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:21.583013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:21.584238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:21.584238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:21.584238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:21.584238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:21.584238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 642/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:21.589792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:21.679142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:21.702209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a722e43-b273-4384-8ff7-c14a67a67b21 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:21.742873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:21.743469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:21.743754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:21.744032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:21.744786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:21.745119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:21.746501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:21.759037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 624/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:21.765135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:21.877726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acde3c-5bf0-455d-ab7d-baa6fc0b488f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:21.898732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:21.914918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:21.915724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:21.916001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:21.916273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:21.924331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:21.925784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:16:21.940254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:21.941273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:21.942281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 643/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:21.947800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:22.053707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd06017-fdf7-4800-ad8a-6a17a808b55d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:22.089277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:22.089943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:22.090222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:22.090493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:22.091357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:22.091661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:22.093082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:22.102262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:22.112353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 625/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Jul 15 15:16:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:22.119675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:22.250773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb174d1-1fa4-4ff2-a568-f03d75d10183 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:22.286231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:22.287154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:22.287547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:22.287822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:22.299313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:22.300772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:22.317946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:22.319090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:22.320089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:22.320089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:22.320089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:22.320089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:22.320089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 644/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:22.325575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:22.429819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891dee2-c467-4f2c-b5d3-35b113fc5a2f None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:22.464689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:22.465354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:22.465632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:22.465904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:22.466656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:22.466963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:22.468366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:22.480769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 626/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:22.487215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:22.524134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:22.612053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110686b-5ead-4b6e-8db7-8ecd14b6f745 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:22.653297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:22.654108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:22.654395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:22.654701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:22.661596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:22.663212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:16:22.680182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:22.681490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:22.682625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 645/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:22.687716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:22.750454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:22.807023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4979db0-4e7f-432f-a315-06de1cf3c8dd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:22.843884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:22.844532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:22.844809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:22.845078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:22.845857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:22.846165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:22.847634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e59f57-0326-4bf4-bd8f-d27a3761a955 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:22.861834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 627/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:22.868656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:22.980097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:22.991406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4ff57-5644-4a91-8681-a9df9d1414fb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:23.029175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef958f4-0bd9-4406-b2f7-a9052fe86044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:23.030075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef958f4-0bd9-4406-b2f7-a9052fe86044 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:23.030355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef958f4-0bd9-4406-b2f7-a9052fe86044 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:23.030671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef958f4-0bd9-4406-b2f7-a9052fe86044 None admin] PATH_INFO: `/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/13549f6dc97241f28501b4c3e3c25959/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:23.032852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef958f4-0bd9-4406-b2f7-a9052fe86044 None admin] RBAC: Authorizing `identity:create_grant(project_id=485b3fa9c86c4ef6be14140d774f5730, user_id=13549f6dc97241f28501b4c3e3c25959, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:23.034310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef958f4-0bd9-4406-b2f7-a9052fe86044 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:23.049106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 646/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Fri Jul 15 15:16:22 2022] PUT /identity/v3/projects/485b3fa9c86c4ef6be14140d774f5730/users/13549f6dc97241f28501b4c3e3c25959/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:16:25.070763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dec45-7772-45d4-a979-56e12ca88906 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:25.071377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dec45-7772-45d4-a979-56e12ca88906 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:25.071377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dec45-7772-45d4-a979-56e12ca88906 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:25.072694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 628/1274] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:25.078957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:25.079414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:25.079695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:25.099274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:25.100475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:25.183745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 647/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:25.206604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dec45-7772-45d4-a979-56e12ca88906 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:25.320886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dec45-7772-45d4-a979-56e12ca88906 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:25.338621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:25.365517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:25.366774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:25.367155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:25.367548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:25.383382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 15:16:25.391163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:25.392260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:25.393213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 629/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Fri Jul 15 15:16:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:25.401154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:25.488039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:25.508351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811cddf-d81a-4d5d-ae27-d4abef6d5b80 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:25.543800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:25.544452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:25.544731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:25.544997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:25.545747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:25.546050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:25.547478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:25.556041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 648/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Fri Jul 15 15:16:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:25.562042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:25.667589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55507-a82d-4950-9eef-452751e19049 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:25.703832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:25.704626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:25.704909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:25.705182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:25.716001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:25.717444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:25.734190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:25.735334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:25.736257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:25.736257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:25.736257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:25.736257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:25.736257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 630/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:25.742277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:25.856132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efe654-1964-4426-823c-0447cae1d0fc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:25.903808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:25.904491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:25.904767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:25.905013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:25.905779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:25.906079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:25.907626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:25.921345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 649/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:25.928510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:25.963326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:26.041021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5176-f3de-4ed3-878f-7b0b60d54f4d None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:26.081193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:26.082126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:26.082472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:26.083018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:26.092343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:26.093793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 15:16:26.108790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:26.109809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:26.111034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 631/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 15:16:26.116280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:26.120288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:26.224235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab62aff-21f1-49e6-9848-0baf28165224 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:26.263365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:26.264037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:26.264319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:26.264595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:26.265433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:26.265742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:26.267196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:26.284596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 650/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Fri Jul 15 15:16:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:26.290889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:26.384328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:26.394638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de411d7-cb1b-41f6-8c1e-7439c6e00506 None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:26.433433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:26.434080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:26.434362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:26.434662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:26.506307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:26.507809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:26.611174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:26.623880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 632/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Fri Jul 15 15:16:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 15:16:26.630555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:26.707754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:26.749301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef5860-2e65-4f65-8086-403b8d44fcaf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:26.786730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:26.787581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:26.787863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:26.788137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:26.798810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:26.800242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 15:16:26.819546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:26.820572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:26.821508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 15:16:26.821508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 15:16:26.821508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 15:16:26.821508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:26.821508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 651/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:26.829624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:26.941255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d68d5-b121-47c4-9841-222a1606927b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:26.980818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:26.981410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:26.981689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:26.981962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:26.982715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:26.983024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:26.984395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:26.997442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 633/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Fri Jul 15 15:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:27.007813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:27.123508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29b5c-f80c-4f2d-8cc2-89f51cd9c0dd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:27.139141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:27.161778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:27.162630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:27.162913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:27.163184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:27.170209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:27.171701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 15:16:27.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:27.187050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Jul 15 15:16:27.188449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 652/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Fri Jul 15 15:16:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:27.198446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:27.264605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:27.319307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a96d0-87bb-49c7-adb7-3d1c600789fd None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:27.360331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:27.360994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:27.361280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:27.361555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:27.362353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:27.362699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:27.364140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d91d3-6d8d-4481-ab05-efb4f6e9b6e6 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:27.376713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 634/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Fri Jul 15 15:16:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:27.383477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:27.498552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732ed65-f1d1-44b4-8528-3d7d659e772b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:27.535074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:27.535636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:27.535920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:27.536195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:27.537569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] RBAC: Adding query filter params (role.id=fb625882070a4b2bb3d8e14f8f69c4d5, scope.project.id=485b3fa9c86c4ef6be14140d774f5730, user.id=13549f6dc97241f28501b4c3e3c25959) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:27.537891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:27.539401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cbfc6-bb3f-47b7-9404-67159464707b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:27.545384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 653/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Fri Jul 15 15:16:27 2022] GET /identity/v3/role_assignments?user.id=13549f6dc97241f28501b4c3e3c25959&scope.project.id=485b3fa9c86c4ef6be14140d774f5730&role.id=fb625882070a4b2bb3d8e14f8f69c4d5 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:29.601918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e12c-28e9-4da5-b35b-6fd1fe01fbbc None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:29.602537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e12c-28e9-4da5-b35b-6fd1fe01fbbc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:29.602537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8e12c-28e9-4da5-b35b-6fd1fe01fbbc None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:29.603883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 635/1288] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:29.610122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:29.611026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:29.611026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:29.635362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:29.637272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:29.673746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:29.719386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 654/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:29.741624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8e12c-28e9-4da5-b35b-6fd1fe01fbbc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:29.855539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8e12c-28e9-4da5-b35b-6fd1fe01fbbc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:29.866450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:29.895898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:29.896714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:29.896993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:29.897271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:29.897924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:29.899327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:29.912158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:29.913401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:29.914792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 636/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:16:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:29.923471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:30.035013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3c4e-e4b9-47b1-b4c7-09f0b82d9aef None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:30.069978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:30.070659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:30.070944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:30.071207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:30.071925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] RBAC: Adding query filter params (name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:30.072230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:30.073615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53554-7e69-4dc1-9868-b3bf8324a9de None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:30.078691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 655/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Jul 15 15:16:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:30.086684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:30.189040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e15897-1b8e-437c-9ffd-e9882ea7584e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:30.225103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:30.225758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:30.226043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:30.226680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] PATH_INFO: `/v3/services` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:30.227384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:30.227690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:30.229061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7ee2a-4b4c-4469-8d5c-3d87d80a9735 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:30.234264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 637/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Jul 15 15:16:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:32.212110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44e60d-fc26-4f0b-b128-b1f8ee6b331b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:32.212676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44e60d-fc26-4f0b-b128-b1f8ee6b331b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:32.212676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44e60d-fc26-4f0b-b128-b1f8ee6b331b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:32.214881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 656/1293] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:32.223651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665d97-8aeb-45b1-ada6-e5fad6663745 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:32.224414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665d97-8aeb-45b1-ada6-e5fad6663745 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:32.224612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665d97-8aeb-45b1-ada6-e5fad6663745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:32.250658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a0665d97-8aeb-45b1-ada6-e5fad6663745 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:32.252611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a0665d97-8aeb-45b1-ada6-e5fad6663745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:32.295165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:32.338265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 638/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:32.372178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44e60d-fc26-4f0b-b128-b1f8ee6b331b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:32.415243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:32.490800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44e60d-fc26-4f0b-b128-b1f8ee6b331b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:32.527229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48b7be-deea-47aa-8289-8e1845b08acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:32.527888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b7be-deea-47aa-8289-8e1845b08acb None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:32.528165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b7be-deea-47aa-8289-8e1845b08acb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:32.528439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b7be-deea-47aa-8289-8e1845b08acb None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:32.529166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48b7be-deea-47aa-8289-8e1845b08acb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:32.530565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48b7be-deea-47aa-8289-8e1845b08acb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:32.554325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 657/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Fri Jul 15 15:16:32 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:16:34.574479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a5110-3b2e-4ef9-a3d6-bbc832a84cb4 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:34.575445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a5110-3b2e-4ef9-a3d6-bbc832a84cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:34.575445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a5110-3b2e-4ef9-a3d6-bbc832a84cb4 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:34.577626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 639/1296] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:34.583232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:34.583736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:34.583736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:34.601443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:34.602697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:34.685617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 658/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:34.719599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a5110-3b2e-4ef9-a3d6-bbc832a84cb4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:34.827619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a5110-3b2e-4ef9-a3d6-bbc832a84cb4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:34.863632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:34.864581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:34.864858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:34.865132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:34.865767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:34.867161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:34.879365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:34.880472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:34.881604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 640/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:16:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:34.887160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:34.908247 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:34.997903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68722b0-cb33-4c99-83b7-927bafe70078 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:35.036378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:35.037067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:35.037330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:35.037607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:35.038311 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:35.038656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:35.040056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:35.046071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 659/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Jul 15 15:16:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:35.052565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:35.086588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:35.162201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec45aa8-4830-42bd-ac6b-bd6fa52ae45e None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:35.196121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:35.196781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:35.197062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:35.197335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:35.198138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] RBAC: Adding query filter params (interface=public, service_id=6d395ded6812408f96efddf0e660847f, region_id=RegionOne) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:35.198447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:35.199866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97927c95-bdfd-4c2f-8a28-afe435883286 None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:35.205055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 641/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Fri Jul 15 15:16:35 2022] GET /identity/v3/endpoints?service_id=6d395ded6812408f96efddf0e660847f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:35.211373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:35.300957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:35.320166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204945d3-025e-4cc2-91f1-f9d52ea353cc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:35.360312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc61ca9-66bb-4007-81e4-d35d05c2910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:35.360889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc61ca9-66bb-4007-81e4-d35d05c2910e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:35.361180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc61ca9-66bb-4007-81e4-d35d05c2910e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:35.361454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc61ca9-66bb-4007-81e4-d35d05c2910e None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:35.362139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc61ca9-66bb-4007-81e4-d35d05c2910e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:35.363558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc61ca9-66bb-4007-81e4-d35d05c2910e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:35.370803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 660/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Fri Jul 15 15:16:35 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:16:37.344969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d73d3-1efe-4f12-8a7e-b2cb07832efb None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:37.345864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d73d3-1efe-4f12-8a7e-b2cb07832efb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:37.345864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d73d3-1efe-4f12-8a7e-b2cb07832efb None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:37.348042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 642/1302] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:16:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:16:37.354210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:37.354649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:37.354798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:37.372532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:16:37.373741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:16:37.446580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 661/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:16:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 15:16:37.469541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d73d3-1efe-4f12-8a7e-b2cb07832efb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:37.577058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:37.587922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d73d3-1efe-4f12-8a7e-b2cb07832efb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:37.627315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:37.628042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:37.628325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:37.628604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:37.629264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:37.630803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 15:16:37.642837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 15:16:37.644036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application with Lock( Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self._enter() Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application return self.creator() Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application created_value = creator( Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: ERROR keystone.server.flask.application Jul 15 15:16:37.645078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 643/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Fri Jul 15 15:16:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 15:16:37.650503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:37.734909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:37.766039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb63d-9c64-4dea-b0c3-1481437a5487 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:37.801723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:37.802381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:37.802685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:37.802956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:37.803653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:16:37.803952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:37.805334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b71570-9e11-4a09-8030-2ce873976296 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:37.810321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 662/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Fri Jul 15 15:16:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:16:37.816594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:16:37.922171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57057055-2d41-412a-8b41-37ea7ff25c4b None admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:16:37.966474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb4cf3-0b53-4d04-8e71-fbee399b583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:16:37.967444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb4cf3-0b53-4d04-8e71-fbee399b583c None admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:16:37.967826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb4cf3-0b53-4d04-8e71-fbee399b583c None admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:16:37.968158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb4cf3-0b53-4d04-8e71-fbee399b583c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:16:37.969306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb4cf3-0b53-4d04-8e71-fbee399b583c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:16:37.971413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb4cf3-0b53-4d04-8e71-fbee399b583c None admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:16:37.993215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 644/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Fri Jul 15 15:16:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:16:47.807766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=30 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:16:47.988516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:41.767334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750030-b0b4-4a5f-b771-a1800c6e2e1e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:41.768378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750030-b0b4-4a5f-b771-a1800c6e2e1e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:41.768870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750030-b0b4-4a5f-b771-a1800c6e2e1e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:41.774629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 663/1307] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:17:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:41.776118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcae1-ed8e-4257-8317-d8e516a4e017 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:41.776553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcae1-ed8e-4257-8317-d8e516a4e017 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:41.776777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bcae1-ed8e-4257-8317-d8e516a4e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:41.929378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-be2bcae1-ed8e-4257-8317-d8e516a4e017 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:41.930833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be2bcae1-ed8e-4257-8317-d8e516a4e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:42.174547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 645/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:42.225303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921ddeb-2cf3-4a90-b12d-0ab7e4a4be6e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:42.226005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921ddeb-2cf3-4a90-b12d-0ab7e4a4be6e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:42.226700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921ddeb-2cf3-4a90-b12d-0ab7e4a4be6e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:42.229023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 664/1309] 10.222.0.40 () {54 vars in 964 bytes} [Fri Jul 15 15:17:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:42.236380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2744180-52ea-4f3e-b0f8-9fb8ef9b51eb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:42.236931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2744180-52ea-4f3e-b0f8-9fb8ef9b51eb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:42.237461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2744180-52ea-4f3e-b0f8-9fb8ef9b51eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:42.392292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c2744180-52ea-4f3e-b0f8-9fb8ef9b51eb None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:42.393541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c2744180-52ea-4f3e-b0f8-9fb8ef9b51eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:42.420098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:42.495376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 646/1310] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Jul 15 15:17:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:42.496424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921ddeb-2cf3-4a90-b12d-0ab7e4a4be6e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:17:42.498200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1921ddeb-2cf3-4a90-b12d-0ab7e4a4be6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:42.608926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921ddeb-2cf3-4a90-b12d-0ab7e4a4be6e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:17:42.685562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:17:42.686331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:42.686657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:42.686936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:42.688595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:42.746580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:42.803511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:17:42.805267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f1304-1206-4b99-9324-eccf97cdb696 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:17:42.826718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 665/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:17:45.171289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a317-bad7-4db2-92a7-c48f6d707cef None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:45.171891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a317-bad7-4db2-92a7-c48f6d707cef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:45.171891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44a317-bad7-4db2-92a7-c48f6d707cef None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:45.174023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 647/1312] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:45.178534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecefa41-41af-4583-8757-d93b58b2ed2f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:45.178980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecefa41-41af-4583-8757-d93b58b2ed2f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:45.179438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecefa41-41af-4583-8757-d93b58b2ed2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:45.262445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6ecefa41-41af-4583-8757-d93b58b2ed2f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:45.263696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ecefa41-41af-4583-8757-d93b58b2ed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:45.367413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 666/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:45.406469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a317-bad7-4db2-92a7-c48f6d707cef None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:17:45.408104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bc44a317-bad7-4db2-92a7-c48f6d707cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:45.535949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44a317-bad7-4db2-92a7-c48f6d707cef None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:17:45.620178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:45.623578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:17:45.624736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:45.625026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:45.625630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:45.627126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:45.741890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:17:45.743783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9da177-59d1-47e6-af03-4e72ef3e9357 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:17:45.764325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 648/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:17:48.130176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbbe7c-a3d0-4dd1-9ab1-1f5bad3441b6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:48.131087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbbe7c-a3d0-4dd1-9ab1-1f5bad3441b6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:48.131087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbbe7c-a3d0-4dd1-9ab1-1f5bad3441b6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:48.133246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 667/1315] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:17:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:48.138887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fc8e9-aed8-4005-8946-3aa88375d6ff None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:48.139314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fc8e9-aed8-4005-8946-3aa88375d6ff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:48.139408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fc8e9-aed8-4005-8946-3aa88375d6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:48.210446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-053fc8e9-aed8-4005-8946-3aa88375d6ff None None] MFA Rules not processed for user `d198e1a875754bcc9e77e87149e160fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:48.211967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-053fc8e9-aed8-4005-8946-3aa88375d6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:48.265815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:48.312732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 649/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:17:48 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:48.353037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbbe7c-a3d0-4dd1-9ab1-1f5bad3441b6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:17:48.355639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-17dbbe7c-a3d0-4dd1-9ab1-1f5bad3441b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:48.372063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:48.486812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbbe7c-a3d0-4dd1-9ab1-1f5bad3441b6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:17:48.566036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:17:48.566719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:48.567000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:48.567239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:48.568918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:48.633030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:48.689598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:17:48.691139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d6700-08ab-4a41-9d32-d9d1d994fc50 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:17:48.710920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 668/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:17:48 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:17:51.387377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa09b1-d957-4274-8bd1-22acfcabd73d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:51.387946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa09b1-d957-4274-8bd1-22acfcabd73d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:51.387946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa09b1-d957-4274-8bd1-22acfcabd73d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:51.389326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 650/1318] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:51.396028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9a5c6-b289-4933-b9c0-b276a7c7b553 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:51.396742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9a5c6-b289-4933-b9c0-b276a7c7b553 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:51.397082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9a5c6-b289-4933-b9c0-b276a7c7b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:51.476622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-13e9a5c6-b289-4933-b9c0-b276a7c7b553 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:51.478061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-13e9a5c6-b289-4933-b9c0-b276a7c7b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:51.614572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 669/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:51.658416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa09b1-d957-4274-8bd1-22acfcabd73d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:17:51.660015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7ffa09b1-d957-4274-8bd1-22acfcabd73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:51.772897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa09b1-d957-4274-8bd1-22acfcabd73d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:17:51.854874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:17:51.855887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:51.856214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:51.856461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:51.858205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:51.866370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:51.996026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:17:51.997478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df2e917-1b58-4338-9756-b20acd8af7bc service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:17:52.020779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 651/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:17:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:17:54.715281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc095e-95a1-497e-92db-28ee7212fe41 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:54.715949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc095e-95a1-497e-92db-28ee7212fe41 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:54.715949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc095e-95a1-497e-92db-28ee7212fe41 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:54.717233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 670/1321] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:54.723010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d5d71-a01c-4779-a375-ae43959bcd03 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:54.723456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d5d71-a01c-4779-a375-ae43959bcd03 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:54.723551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d5d71-a01c-4779-a375-ae43959bcd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:54.792608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b28d5d71-a01c-4779-a375-ae43959bcd03 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:54.793905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b28d5d71-a01c-4779-a375-ae43959bcd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:54.858331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:54.900082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 652/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:54.940791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc095e-95a1-497e-92db-28ee7212fe41 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:17:54.942985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-26fc095e-95a1-497e-92db-28ee7212fe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:54.966825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:55.066047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc095e-95a1-497e-92db-28ee7212fe41 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:17:55.148414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:17:55.149097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:55.149380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:55.149656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:55.151355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:55.226611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:55.266211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:17:55.267723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0ab99e-ebdf-4f7e-a6ee-fbac0920d2b2 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:17:55.286425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 671/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:17:57.420408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4c2eb-f58a-4b40-bb71-16207b133bdd None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:57.421039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4c2eb-f58a-4b40-bb71-16207b133bdd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:57.421039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4c2eb-f58a-4b40-bb71-16207b133bdd None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:57.422536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 653/1324] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:17:57.428046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65947f00-1ba4-445a-9a2f-50fad1d27222 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:57.428488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65947f00-1ba4-445a-9a2f-50fad1d27222 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:57.428769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65947f00-1ba4-445a-9a2f-50fad1d27222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:57.502592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-65947f00-1ba4-445a-9a2f-50fad1d27222 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:17:57.503855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-65947f00-1ba4-445a-9a2f-50fad1d27222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:57.609008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 672/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:17:57.644915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4c2eb-f58a-4b40-bb71-16207b133bdd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:17:57.646585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8ed4c2eb-f58a-4b40-bb71-16207b133bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:57.767007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed4c2eb-f58a-4b40-bb71-16207b133bdd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:17:57.842978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:17:57.843782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:17:57.844069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:17:57.844345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:17:57.846381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:17:57.865567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:17:57.972196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:17:57.973646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22996bcc-cf47-4e1e-9d6e-397b925af150 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:17:57.994837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 654/1326] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:00.044721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f1fa-478b-47bc-b8ed-58510f847f83 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:00.045298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f1fa-478b-47bc-b8ed-58510f847f83 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:00.045298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72f1fa-478b-47bc-b8ed-58510f847f83 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:00.046940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 673/1327] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:00.052159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8f367-0e14-495f-b167-bdf394172617 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:00.052646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8f367-0e14-495f-b167-bdf394172617 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:00.052755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8f367-0e14-495f-b167-bdf394172617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:00.121859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dce8f367-0e14-495f-b167-bdf394172617 None None] MFA Rules not processed for user `d198e1a875754bcc9e77e87149e160fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:00.123042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dce8f367-0e14-495f-b167-bdf394172617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:00.193786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:00.230353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 655/1328] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:00.269071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72f1fa-478b-47bc-b8ed-58510f847f83 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:00.270686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bd72f1fa-478b-47bc-b8ed-58510f847f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:00.305463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:00.382213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72f1fa-478b-47bc-b8ed-58510f847f83 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:00.479760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:00.480423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:00.480704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:00.480982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:00.482963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:00.560143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:00.592457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:00.593943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119374bf-59a9-415a-a426-c6ace30b43f9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:00.612645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 674/1329] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:02.892991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af8c45-749c-4a2b-a386-94bd18a72868 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:02.893591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af8c45-749c-4a2b-a386-94bd18a72868 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:02.893591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af8c45-749c-4a2b-a386-94bd18a72868 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:02.894956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 656/1330] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:02.901381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb3501-5b77-4ead-82ab-1b93578f4f94 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:02.901837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb3501-5b77-4ead-82ab-1b93578f4f94 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:02.901934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb3501-5b77-4ead-82ab-1b93578f4f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:02.979791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-febb3501-5b77-4ead-82ab-1b93578f4f94 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:02.981000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-febb3501-5b77-4ead-82ab-1b93578f4f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:03.080787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 675/1331] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:03.117232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af8c45-749c-4a2b-a386-94bd18a72868 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:03.118804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-50af8c45-749c-4a2b-a386-94bd18a72868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:03.232060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af8c45-749c-4a2b-a386-94bd18a72868 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:03.308916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:03.309570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:03.309848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:03.310126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:03.311794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:03.339556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:03.436978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:03.438417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912812bf-9973-4ead-9f08-7b499b567770 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:03.458093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 657/1332] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:05.655501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd7438-9149-4825-9fd7-3d9d55956c66 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:05.656138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd7438-9149-4825-9fd7-3d9d55956c66 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:05.656138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd7438-9149-4825-9fd7-3d9d55956c66 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:05.657528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 676/1333] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:05.665370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7521895-03e7-4a68-b4a1-14abfc1c9cc1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:05.665873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7521895-03e7-4a68-b4a1-14abfc1c9cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:05.665873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7521895-03e7-4a68-b4a1-14abfc1c9cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:05.744535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c7521895-03e7-4a68-b4a1-14abfc1c9cc1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:05.745743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c7521895-03e7-4a68-b4a1-14abfc1c9cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:05.844232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:05.867695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 658/1334] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:05.927373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5170f7f-bcc6-402d-bd2d-aca575524891 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:05.927781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5170f7f-bcc6-402d-bd2d-aca575524891 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:05.928125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5170f7f-bcc6-402d-bd2d-aca575524891 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:05.930243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 677/1335] 10.222.0.40 () {54 vars in 964 bytes} [Fri Jul 15 15:18:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:05.936858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ba191-6baf-470d-ba30-234e804cd517 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:05.937158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ba191-6baf-470d-ba30-234e804cd517 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:05.937442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ba191-6baf-470d-ba30-234e804cd517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:06.018487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-099ba191-6baf-470d-ba30-234e804cd517 None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:06.019750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-099ba191-6baf-470d-ba30-234e804cd517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:06.102192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 659/1336] 10.222.0.40 () {58 vars in 1061 bytes} [Fri Jul 15 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:06.111910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5170f7f-bcc6-402d-bd2d-aca575524891 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:06.113548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d5170f7f-bcc6-402d-bd2d-aca575524891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:06.160163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:06.231241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5170f7f-bcc6-402d-bd2d-aca575524891 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:06.311245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:06.311871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:06.312132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:06.312426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:06.314096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:06.407669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:06.433973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:06.435430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38336fe7-984e-40d3-8105-a5c07a64dc13 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:06.455037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 678/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:09.324214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41050b-16af-449c-8136-d2eef5651868 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:09.325101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41050b-16af-449c-8136-d2eef5651868 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:09.325101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41050b-16af-449c-8136-d2eef5651868 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:09.327356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 660/1338] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:09.336791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa800f1c-cbb8-424c-a696-33fc59173a6b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:09.337218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa800f1c-cbb8-424c-a696-33fc59173a6b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:09.337346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa800f1c-cbb8-424c-a696-33fc59173a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:09.411037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fa800f1c-cbb8-424c-a696-33fc59173a6b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:09.412517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fa800f1c-cbb8-424c-a696-33fc59173a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:09.514548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 679/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:09.551420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41050b-16af-449c-8136-d2eef5651868 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:09.553016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ca41050b-16af-449c-8136-d2eef5651868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:09.630083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:09.658856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41050b-16af-449c-8136-d2eef5651868 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:09.737458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:09.738108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:09.738394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:09.738750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:09.740399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:09.853923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:09.855413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa36ea-0674-472a-9b28-1f24ca3f3eed service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:09.874029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 661/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:14.065859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234c807-21b5-4f88-8fe9-354078db4cd8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:14.066346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234c807-21b5-4f88-8fe9-354078db4cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:14.066679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234c807-21b5-4f88-8fe9-354078db4cd8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:14.068027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 680/1341] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:14.073464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6548-6e03-43ac-99e6-06fda3a1b86c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:14.073869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6548-6e03-43ac-99e6-06fda3a1b86c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:14.074017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb6548-6e03-43ac-99e6-06fda3a1b86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:14.078309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:14.139493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-90fb6548-6e03-43ac-99e6-06fda3a1b86c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:14.140615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-90fb6548-6e03-43ac-99e6-06fda3a1b86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:14.244066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 662/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:14.282668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234c807-21b5-4f88-8fe9-354078db4cd8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:14.284218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8234c807-21b5-4f88-8fe9-354078db4cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:14.332845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:14.392781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234c807-21b5-4f88-8fe9-354078db4cd8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:14.467663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:14.468317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:14.468595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:14.468868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:14.470547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:14.568311 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:14.585822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:14.587322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77461b2a-6ea0-4576-816c-f2c3e6402cb3 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:14.606564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 681/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:17.221006 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfc42f-c890-4cd3-a72a-d7588872cb9d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:17.221633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfc42f-c890-4cd3-a72a-d7588872cb9d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:17.221633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfc42f-c890-4cd3-a72a-d7588872cb9d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:17.223087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 663/1344] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:17.229126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ebc0c-a240-4d1e-991a-37fd31fae397 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:17.229725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ebc0c-a240-4d1e-991a-37fd31fae397 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:17.229725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ebc0c-a240-4d1e-991a-37fd31fae397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:17.302260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-903ebc0c-a240-4d1e-991a-37fd31fae397 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:17.303562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-903ebc0c-a240-4d1e-991a-37fd31fae397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:17.411666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 682/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:17.449540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfc42f-c890-4cd3-a72a-d7588872cb9d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:17.451178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b4dfc42f-c890-4cd3-a72a-d7588872cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:17.542001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:17.559484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfc42f-c890-4cd3-a72a-d7588872cb9d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:17.634712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:17.635280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:17.635560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:17.635832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:17.637468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:17.754015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:17.755477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71bd1d-286b-4f8a-a949-75bf0e0ad5bc service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:17.773786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 664/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:22.046037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f7b30-9397-4fb9-b328-37e74c673377 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:22.046718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f7b30-9397-4fb9-b328-37e74c673377 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:22.046718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f7b30-9397-4fb9-b328-37e74c673377 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:22.047995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 683/1347] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:22.054212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790913d-f145-461a-b073-34b47a157996 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:22.054791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790913d-f145-461a-b073-34b47a157996 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:22.054791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790913d-f145-461a-b073-34b47a157996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:22.071199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:22.132078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4790913d-f145-461a-b073-34b47a157996 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:22.133153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4790913d-f145-461a-b073-34b47a157996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:22.236590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 665/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:22.274494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f7b30-9397-4fb9-b328-37e74c673377 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:22.276136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-657f7b30-9397-4fb9-b328-37e74c673377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:22.339193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:22.391715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f7b30-9397-4fb9-b328-37e74c673377 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:22.478386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:22.479075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:22.479332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:22.479604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:22.481279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:22.603152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:22.606994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:22.608773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c60cc5-07e8-4208-bc88-6e7e544c83ff service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:22.629061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 684/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:27.606956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb2e73-8ee7-4733-8227-c6341c0f5a90 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:27.607526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb2e73-8ee7-4733-8227-c6341c0f5a90 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:27.607526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb2e73-8ee7-4733-8227-c6341c0f5a90 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:27.608885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 666/1350] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:27.615261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26802671-6527-49be-a154-9edcadf75efb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:27.615751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26802671-6527-49be-a154-9edcadf75efb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:27.615751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26802671-6527-49be-a154-9edcadf75efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:27.713667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-26802671-6527-49be-a154-9edcadf75efb None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:27.715250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-26802671-6527-49be-a154-9edcadf75efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:27.826182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 685/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:31.746036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716948d-c1c8-4350-829c-9de9027df67a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:31.746904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716948d-c1c8-4350-829c-9de9027df67a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:31.746904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716948d-c1c8-4350-829c-9de9027df67a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:31.748295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 667/1352] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:31.753845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd207ef-870c-4bac-9a19-d9208c83d1dc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:31.754699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd207ef-870c-4bac-9a19-d9208c83d1dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:31.755211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd207ef-870c-4bac-9a19-d9208c83d1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:31.811557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:31.839395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2bd207ef-870c-4bac-9a19-d9208c83d1dc None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:31.841022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2bd207ef-870c-4bac-9a19-d9208c83d1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:31.963963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 686/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:32.234876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=59 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:33.448666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521673b-d227-458d-9a86-320c447f03c2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:33.449296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521673b-d227-458d-9a86-320c447f03c2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:33.449296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521673b-d227-458d-9a86-320c447f03c2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:33.450594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 668/1354] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:33.456006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650f00-a04e-4c45-858c-581b4b21fd9e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:33.456427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650f00-a04e-4c45-858c-581b4b21fd9e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:33.456599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650f00-a04e-4c45-858c-581b4b21fd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:33.542298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-23650f00-a04e-4c45-858c-581b4b21fd9e None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:33.543573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-23650f00-a04e-4c45-858c-581b4b21fd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:33.586799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:33.655537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 687/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:18:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:37.221543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b1335-387b-4b11-84c3-5e2e93a7a3a9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:37.222508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b1335-387b-4b11-84c3-5e2e93a7a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:37.223067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28b1335-387b-4b11-84c3-5e2e93a7a3a9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:37.224938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 669/1356] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:37.230600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30011412-d046-462d-8b64-5b33d3f33bc9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:37.231584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30011412-d046-462d-8b64-5b33d3f33bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:37.232088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30011412-d046-462d-8b64-5b33d3f33bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:37.306262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-30011412-d046-462d-8b64-5b33d3f33bc9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:37.307554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-30011412-d046-462d-8b64-5b33d3f33bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:37.423066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 688/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:37.473871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac5941-0892-4f09-8bdd-76a4a7b77d5c None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:37.478259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac5941-0892-4f09-8bdd-76a4a7b77d5c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:37.478736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac5941-0892-4f09-8bdd-76a4a7b77d5c None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:37.480957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 670/1358] 10.222.0.40 () {54 vars in 961 bytes} [Fri Jul 15 15:18:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:37.490159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06624dc-4977-40b2-8af3-fd9924244141 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:37.490399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06624dc-4977-40b2-8af3-fd9924244141 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:37.492821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06624dc-4977-40b2-8af3-fd9924244141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:37.511192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:37.598098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b06624dc-4977-40b2-8af3-fd9924244141 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:37.599634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b06624dc-4977-40b2-8af3-fd9924244141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:37.723575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 689/1359] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Jul 15 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:37.731457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac5941-0892-4f09-8bdd-76a4a7b77d5c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:37.732942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-88ac5941-0892-4f09-8bdd-76a4a7b77d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:37.892819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac5941-0892-4f09-8bdd-76a4a7b77d5c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:37.982749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:38.026584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:38.027253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:38.027539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:38.027802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:38.029465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:38.131766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:38.133467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb3969-a018-447d-a59b-5e3c866e10db service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:38.154545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 671/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:40.385297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5ab8d-5b99-4d92-99a3-30e399077c66 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:40.385916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5ab8d-5b99-4d92-99a3-30e399077c66 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:40.385916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5ab8d-5b99-4d92-99a3-30e399077c66 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:40.387438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 690/1361] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:40.392771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c524-ba05-43dc-b3d2-221dfcc67519 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:40.393241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c524-ba05-43dc-b3d2-221dfcc67519 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:40.393346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9c524-ba05-43dc-b3d2-221dfcc67519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:40.461604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-82d9c524-ba05-43dc-b3d2-221dfcc67519 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:40.463268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-82d9c524-ba05-43dc-b3d2-221dfcc67519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:40.502296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:40.568770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 672/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:40.618367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5ab8d-5b99-4d92-99a3-30e399077c66 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:40.620179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-64d5ab8d-5b99-4d92-99a3-30e399077c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:40.716168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:40.810220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5ab8d-5b99-4d92-99a3-30e399077c66 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:40.974404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:40.975106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:40.975455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:40.975820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:40.977557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:41.002133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:41.074573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:41.076244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d2fb6-e5c9-43c4-aa5b-1f6108ad4fb5 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:41.095603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 691/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:43.427697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4568-bb34-463a-ba27-784a17981551 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:43.428296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4568-bb34-463a-ba27-784a17981551 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:43.428296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4568-bb34-463a-ba27-784a17981551 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:43.429655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 673/1364] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:43.435369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219a99a-d8fa-4a3e-ab1d-a4472ce2d719 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:43.435927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219a99a-d8fa-4a3e-ab1d-a4472ce2d719 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:43.435927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219a99a-d8fa-4a3e-ab1d-a4472ce2d719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:43.512084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7219a99a-d8fa-4a3e-ab1d-a4472ce2d719 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:43.513424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7219a99a-d8fa-4a3e-ab1d-a4472ce2d719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:43.616041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:43.618874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 692/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:18:43.665401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bc5b1-7cbe-4f0c-bead-daf4080159e5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:43.665655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bc5b1-7cbe-4f0c-bead-daf4080159e5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:43.666226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8bc5b1-7cbe-4f0c-bead-daf4080159e5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:43.667687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 674/1366] 10.222.0.40 () {54 vars in 961 bytes} [Fri Jul 15 15:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:43.673834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dfc9e-ac06-4b4d-aeba-30670ecf9d96 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:43.674123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dfc9e-ac06-4b4d-aeba-30670ecf9d96 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:43.674398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dfc9e-ac06-4b4d-aeba-30670ecf9d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:43.758094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-636dfc9e-ac06-4b4d-aeba-30670ecf9d96 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:43.759399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-636dfc9e-ac06-4b4d-aeba-30670ecf9d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:43.896865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 693/1367] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Jul 15 15:18:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:43.908871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bc5b1-7cbe-4f0c-bead-daf4080159e5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:43.912860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9c8bc5b1-7cbe-4f0c-bead-daf4080159e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:44.081798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8bc5b1-7cbe-4f0c-bead-daf4080159e5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:44.103409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:44.202536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:44.203229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:44.203547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:44.203826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:44.205523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:44.307286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:44.308958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab66d3-8709-465f-a3b4-a37f6147bc31 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:44.329226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 675/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:45.300570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5de92d-c075-4017-ae47-ab4b4b76d226 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:45.301178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5de92d-c075-4017-ae47-ab4b4b76d226 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:45.301178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5de92d-c075-4017-ae47-ab4b4b76d226 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:45.302761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 694/1369] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:45.308110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867b33-fd9d-4fa6-ac71-44e15205a244 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:45.308559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867b33-fd9d-4fa6-ac71-44e15205a244 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:45.308708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7867b33-fd9d-4fa6-ac71-44e15205a244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:45.344004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:45.384031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c7867b33-fd9d-4fa6-ac71-44e15205a244 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:45.385256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c7867b33-fd9d-4fa6-ac71-44e15205a244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:45.494574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 676/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:45.858790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6b857-5172-4e4d-857d-4620f5adb947 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:45.858790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6b857-5172-4e4d-857d-4620f5adb947 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:45.858790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6b857-5172-4e4d-857d-4620f5adb947 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:45.859998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 695/1371] 10.222.0.40 () {54 vars in 963 bytes} [Fri Jul 15 15:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:45.864202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12c1e-a0e4-4b2a-86a1-1e566b84ec6f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:45.864460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12c1e-a0e4-4b2a-86a1-1e566b84ec6f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:45.864763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c12c1e-a0e4-4b2a-86a1-1e566b84ec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:45.954493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d3c12c1e-a0e4-4b2a-86a1-1e566b84ec6f None None] MFA Rules not processed for user `4cfd0067b59048f1850e8eb6e94640ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:45.955761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d3c12c1e-a0e4-4b2a-86a1-1e566b84ec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:46.006174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:46.039189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 677/1372] 10.222.0.40 () {58 vars in 1060 bytes} [Fri Jul 15 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:18:46.046949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6b857-5172-4e4d-857d-4620f5adb947 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:46.050620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-31e6b857-5172-4e4d-857d-4620f5adb947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:46.107449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:46.181780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6b857-5172-4e4d-857d-4620f5adb947 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:46.267804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:46.268846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:46.269377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:46.269752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:46.271629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:46.390136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:46.408878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:46.408878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2f9a5-5302-498f-84f1-c386baa30690 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:46.440435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf311cdb-a534-4a66-9544-9bafacfa5633 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:46.440952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf311cdb-a534-4a66-9544-9bafacfa5633 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:46.441450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf311cdb-a534-4a66-9544-9bafacfa5633 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:46.441858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 696/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:46.449423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 678/1374] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:46 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:18:46.459370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fa11a-4299-47d1-9210-5bf0be4dc258 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:46.459700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fa11a-4299-47d1-9210-5bf0be4dc258 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:46.459983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fa11a-4299-47d1-9210-5bf0be4dc258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:46.555695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-312fa11a-4299-47d1-9210-5bf0be4dc258 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:46.557162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-312fa11a-4299-47d1-9210-5bf0be4dc258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:46.693387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 697/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:46.730673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf311cdb-a534-4a66-9544-9bafacfa5633 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:46.732554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cf311cdb-a534-4a66-9544-9bafacfa5633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:46.916443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf311cdb-a534-4a66-9544-9bafacfa5633 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:46.970951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:47.034407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:47.035082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:47.035351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:47.035623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:47.037263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:47.150949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:47.152584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c4059-9bc5-4e25-9557-0a30c8da3f64 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:47.172055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 679/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:49.184888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d678200-45fa-4230-898d-9b88c4346670 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:49.185784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d678200-45fa-4230-898d-9b88c4346670 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:49.186063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d678200-45fa-4230-898d-9b88c4346670 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:49.187522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 698/1377] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:49.192918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550fa6-f2bb-4b6b-943b-f34e3b373b8b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:49.193345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550fa6-f2bb-4b6b-943b-f34e3b373b8b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:49.193454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550fa6-f2bb-4b6b-943b-f34e3b373b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:49.265618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-32550fa6-f2bb-4b6b-943b-f34e3b373b8b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:49.266977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32550fa6-f2bb-4b6b-943b-f34e3b373b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:49.273554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:49.371928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 680/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:49.424598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d678200-45fa-4230-898d-9b88c4346670 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:49.426587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2d678200-45fa-4230-898d-9b88c4346670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:49.492194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:49.580641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d678200-45fa-4230-898d-9b88c4346670 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:49.706564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:49.707277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:49.707554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:49.707822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:49.709482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:49.729793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:49.821348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:49.823024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892fd3f-33d3-449d-85d8-fbe42911d3f5 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:49.855341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 699/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:51.951637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f4afc-835e-4be9-b802-261bbfb41d11 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:51.952176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f4afc-835e-4be9-b802-261bbfb41d11 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:51.952176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f4afc-835e-4be9-b802-261bbfb41d11 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:51.953956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 681/1380] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:51.959341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab02eb-bbf8-4a29-bbf6-bb8d83c23e85 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:51.960186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab02eb-bbf8-4a29-bbf6-bb8d83c23e85 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:51.960266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab02eb-bbf8-4a29-bbf6-bb8d83c23e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:52.030090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5eab02eb-bbf8-4a29-bbf6-bb8d83c23e85 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:52.031429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5eab02eb-bbf8-4a29-bbf6-bb8d83c23e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:52.135651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:52.144539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 700/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:52.191593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f4afc-835e-4be9-b802-261bbfb41d11 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:52.193224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-479f4afc-835e-4be9-b802-261bbfb41d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:52.347567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:52.359446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f4afc-835e-4be9-b802-261bbfb41d11 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:52.479575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:52.480449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:52.480736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:52.481020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:52.482751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:52.589466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:52.591284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9df3d-29ba-4271-9965-7781d9e530a6 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:52.602581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:52.612677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 682/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:54.706006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f315bcf-a41e-40c6-bec6-88a9e6dcee59 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:54.706801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f315bcf-a41e-40c6-bec6-88a9e6dcee59 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:54.706801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f315bcf-a41e-40c6-bec6-88a9e6dcee59 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:54.708167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 701/1383] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:18:54.714113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e3563-23ff-4d04-bc17-389513edf055 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:54.714596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e3563-23ff-4d04-bc17-389513edf055 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:54.714771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8e3563-23ff-4d04-bc17-389513edf055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:54.784350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5b8e3563-23ff-4d04-bc17-389513edf055 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:54.785634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5b8e3563-23ff-4d04-bc17-389513edf055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:54.913816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 683/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:54.958711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f315bcf-a41e-40c6-bec6-88a9e6dcee59 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:54.960595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f315bcf-a41e-40c6-bec6-88a9e6dcee59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:55.121872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f315bcf-a41e-40c6-bec6-88a9e6dcee59 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:55.192459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:55.251787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:55.252601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:55.252972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:55.253420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:55.255672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:55.387885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:55.390087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8cd4-5104-485f-b1e1-2d51212ff477 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:55.411304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 702/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:18:57.738894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffc276-b8ba-4a91-bb45-fb9a122b45da None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:57.739517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffc276-b8ba-4a91-bb45-fb9a122b45da None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:57.739517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffc276-b8ba-4a91-bb45-fb9a122b45da None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:57.740937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 684/1386] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:18:57.747208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6330f2-51a7-4f4f-9e91-218575a6cd6b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:57.747709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6330f2-51a7-4f4f-9e91-218575a6cd6b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:57.748005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6330f2-51a7-4f4f-9e91-218575a6cd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:57.836614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0c6330f2-51a7-4f4f-9e91-218575a6cd6b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:18:57.839003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0c6330f2-51a7-4f4f-9e91-218575a6cd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:57.878986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:57.961077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 703/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:18:58.009097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffc276-b8ba-4a91-bb45-fb9a122b45da None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:18:58.010980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b4ffc276-b8ba-4a91-bb45-fb9a122b45da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:58.093626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:58.172397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffc276-b8ba-4a91-bb45-fb9a122b45da None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:18:58.292780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:18:58.293373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:18:58.293617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:18:58.293939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:18:58.295672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:18:58.342002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:18:58.416782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:18:58.419489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9299a7-a4c7-42fc-944b-ca3e97ababf4 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:18:58.443416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 685/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:00.628648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5e1f-cb8f-4b38-87d6-ebcf64ff0e4a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:00.629277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5e1f-cb8f-4b38-87d6-ebcf64ff0e4a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:00.629277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb5e1f-cb8f-4b38-87d6-ebcf64ff0e4a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:00.630584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 704/1389] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:19:00.636169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33400955-76d7-4edb-b030-7d90e8efbd30 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:00.636620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33400955-76d7-4edb-b030-7d90e8efbd30 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:00.636778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33400955-76d7-4edb-b030-7d90e8efbd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:00.708485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-33400955-76d7-4edb-b030-7d90e8efbd30 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:00.709802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-33400955-76d7-4edb-b030-7d90e8efbd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:00.819667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 686/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:00.864856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb5e1f-cb8f-4b38-87d6-ebcf64ff0e4a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:00.866404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1ecb5e1f-cb8f-4b38-87d6-ebcf64ff0e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:01.020120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb5e1f-cb8f-4b38-87d6-ebcf64ff0e4a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:01.038372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:01.144667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:01.145343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:01.145619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:01.145890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:01.147768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:01.241357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:01.243020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cabd6b-506f-4869-a2e4-9b8172338106 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:01.261903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 705/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:01.623289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97f0f5-29bf-44c6-b638-4db64a15e6c8 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:01.623539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97f0f5-29bf-44c6-b638-4db64a15e6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:01.623723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca97f0f5-29bf-44c6-b638-4db64a15e6c8 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:01.625120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 687/1392] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:01.630573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf58af-1372-4c19-9056-959646fa79a5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:01.631069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf58af-1372-4c19-9056-959646fa79a5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:01.631172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf58af-1372-4c19-9056-959646fa79a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:01.656326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:01.700488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9abf58af-1372-4c19-9056-959646fa79a5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:01.702134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9abf58af-1372-4c19-9056-959646fa79a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:01.803325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 706/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:01.861488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e6bf6-08d1-4015-a108-1e22ed6d2f23 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:01.861985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e6bf6-08d1-4015-a108-1e22ed6d2f23 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:01.862085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e6bf6-08d1-4015-a108-1e22ed6d2f23 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:01.863448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 688/1394] 10.222.0.40 () {54 vars in 960 bytes} [Fri Jul 15 15:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:01.868544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd103-b1eb-41b2-bd91-f32d8a6f9822 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:01.868908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd103-b1eb-41b2-bd91-f32d8a6f9822 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:01.869251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd103-b1eb-41b2-bd91-f32d8a6f9822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:01.953917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-549dd103-b1eb-41b2-bd91-f32d8a6f9822 None None] MFA Rules not processed for user `9fe7e2ad26c8483e8aecb89225f3b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:01.955229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-549dd103-b1eb-41b2-bd91-f32d8a6f9822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:01.999924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:02.036996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 707/1395] 10.222.0.40 () {58 vars in 1057 bytes} [Fri Jul 15 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:02.044745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e6bf6-08d1-4015-a108-1e22ed6d2f23 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:02.046279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9f1e6bf6-08d1-4015-a108-1e22ed6d2f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:02.060665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:02.154143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e6bf6-08d1-4015-a108-1e22ed6d2f23 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:02.231164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:02.231987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:02.232264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:02.232538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:02.234191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:02.301794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:02.343908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:02.345359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be098e48-c89e-4888-ac4d-53b73b9aa6d0 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:02.365805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 689/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:03.098042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db94f9-1a3f-47b7-8919-5f2589360d11 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.098636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db94f9-1a3f-47b7-8919-5f2589360d11 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.098748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db94f9-1a3f-47b7-8919-5f2589360d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.171231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b4db94f9-1a3f-47b7-8919-5f2589360d11 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:03.172474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b4db94f9-1a3f-47b7-8919-5f2589360d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.264358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 708/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:03.317165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435239-0be3-4a04-9d80-63b8043530f3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.317704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435239-0be3-4a04-9d80-63b8043530f3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.317812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de435239-0be3-4a04-9d80-63b8043530f3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.319364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 690/1398] 10.222.0.40 () {54 vars in 942 bytes} [Fri Jul 15 15:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:03.320190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91c84f-a89f-4065-9156-31eea85e4c89 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.320484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91c84f-a89f-4065-9156-31eea85e4c89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.320808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91c84f-a89f-4065-9156-31eea85e4c89 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.322392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 709/1399] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:19:03.327188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.327551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.327895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.327996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.328182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.328458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.399839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:03.401328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.405471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] MFA Rules not processed for user `19b3f6ab1b924bb5907bbb33db1a5b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:03.406699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.408269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:03.504746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 710/1400] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:03.520556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 691/1401] 10.222.0.40 () {58 vars in 1039 bytes} [Fri Jul 15 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:03.530354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:03.531688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.550364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:03.551627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.554948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:03.640699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132ccfa-d406-47bc-a4a1-a3ac99bd3dcb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:03.653519 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47255f84-71d7-4e8b-a9fb-ea498b705189 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:03.680616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:03.751040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:03.751706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.751988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.752262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.754038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.775325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:03.775964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:03.776340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:03.776514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:03.778198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:03.791771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:03.840840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:03.842552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ec07e-e5fc-4fb5-97ad-7b81328c0d14 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:03.851443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 711/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:19:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:19:03.876248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:03.877905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fa24f-d6a0-4002-8515-fdcf963d535b service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:03.899063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 692/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:19:06.000085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51639a-3d4d-43aa-82f3-451658e32eb5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:06.000710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51639a-3d4d-43aa-82f3-451658e32eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:06.000710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51639a-3d4d-43aa-82f3-451658e32eb5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:06.002184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 712/1404] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:06.007270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc622b-b737-49e0-8b39-de5c32797af2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:06.007685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc622b-b737-49e0-8b39-de5c32797af2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:06.007869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc622b-b737-49e0-8b39-de5c32797af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:06.082333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d3bc622b-b737-49e0-8b39-de5c32797af2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:06.083727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d3bc622b-b737-49e0-8b39-de5c32797af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:06.188208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:06.200852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 693/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:06.253828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51639a-3d4d-43aa-82f3-451658e32eb5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:06.255406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8b51639a-3d4d-43aa-82f3-451658e32eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:06.333704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:06.424960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51639a-3d4d-43aa-82f3-451658e32eb5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:06.565937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:06.566657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:06.567400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:06.567674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:06.569371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:06.610731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:06.694564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:06.696863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1928421-5ad6-4b3c-bf2b-d61c3e43c5f4 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:06.722371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 713/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:08.884100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbc8f6-ecb8-4628-8dde-9662a4428894 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:08.884961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbc8f6-ecb8-4628-8dde-9662a4428894 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:08.884961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbc8f6-ecb8-4628-8dde-9662a4428894 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:08.886495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 694/1407] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:08.891781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745770af-e0c7-46ac-b263-d3977a5e4ff0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:08.892649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745770af-e0c7-46ac-b263-d3977a5e4ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:08.893140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745770af-e0c7-46ac-b263-d3977a5e4ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:08.975602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-745770af-e0c7-46ac-b263-d3977a5e4ff0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:08.976861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-745770af-e0c7-46ac-b263-d3977a5e4ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:09.102774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 714/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:09.147228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbc8f6-ecb8-4628-8dde-9662a4428894 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:09.148829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-efcbc8f6-ecb8-4628-8dde-9662a4428894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:09.315231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbc8f6-ecb8-4628-8dde-9662a4428894 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:09.389840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:09.449069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:09.450143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:09.450421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:09.450731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:09.452416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:09.554163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:09.556059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359397c4-b2c5-463e-9373-c5be424057cb service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:09.576405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 695/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:19.098167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:19.572779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=66 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:26.144599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14876dc-3ec1-4a6d-9a89-517538981619 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:26.145538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14876dc-3ec1-4a6d-9a89-517538981619 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:26.145799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14876dc-3ec1-4a6d-9a89-517538981619 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:26.147231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 715/1410] 10.222.0.40 () {54 vars in 918 bytes} [Fri Jul 15 15:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:26.153983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6665-8de5-43ce-9402-93d7220f4dc9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:26.154452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6665-8de5-43ce-9402-93d7220f4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:26.154570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6665-8de5-43ce-9402-93d7220f4dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:26.235143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-903f6665-8de5-43ce-9402-93d7220f4dc9 None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:26.236352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-903f6665-8de5-43ce-9402-93d7220f4dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:26.366496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 696/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Jul 15 15:19:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:26.382054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f554b-fa60-46de-ad75-63344e787a86 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:26.382402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f554b-fa60-46de-ad75-63344e787a86 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:26.382875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f554b-fa60-46de-ad75-63344e787a86 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:26.384255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 716/1412] 10.222.0.40 () {54 vars in 918 bytes} [Fri Jul 15 15:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:26.389413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf4f6-bc19-487c-a997-ee435b7ba222 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:26.389708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf4f6-bc19-487c-a997-ee435b7ba222 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:26.389981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcf4f6-bc19-487c-a997-ee435b7ba222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:26.467638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:26.470767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-00fcf4f6-bc19-487c-a997-ee435b7ba222 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:26.472751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-00fcf4f6-bc19-487c-a997-ee435b7ba222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:26.596128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 697/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Fri Jul 15 15:19:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:29.107734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3db02-8132-4614-b762-57eb81d529cc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:29.108637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3db02-8132-4614-b762-57eb81d529cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:29.108637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3db02-8132-4614-b762-57eb81d529cc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:29.110907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:29.116607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbb830-5f32-4149-8613-d13aaaa2b0cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:29.117033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbb830-5f32-4149-8613-d13aaaa2b0cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:29.117357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbb830-5f32-4149-8613-d13aaaa2b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:29.186064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-39dbb830-5f32-4149-8613-d13aaaa2b0cb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:29.187316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-39dbb830-5f32-4149-8613-d13aaaa2b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:29.246583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:29.290803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:29.336336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3db02-8132-4614-b762-57eb81d529cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:29.338146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f5c3db02-8132-4614-b762-57eb81d529cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:29.489884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3db02-8132-4614-b762-57eb81d529cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:29.565283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:29.609802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:29.610451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:29.610764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:29.611036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:29.612680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:29.706270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:29.707927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73e1c8-7940-42e8-b3b5-90bc449b2839 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:29.726204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 718/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:30.655424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78599fb-f383-4a4b-b541-9518a0d20795 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:30.656358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78599fb-f383-4a4b-b541-9518a0d20795 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:30.656358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78599fb-f383-4a4b-b541-9518a0d20795 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:30.658616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:30.667048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343919e-f41e-4e8d-af72-97a5ecd00abd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:30.667495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343919e-f41e-4e8d-af72-97a5ecd00abd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:30.667610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343919e-f41e-4e8d-af72-97a5ecd00abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:30.764519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9343919e-f41e-4e8d-af72-97a5ecd00abd None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:30.765747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9343919e-f41e-4e8d-af72-97a5ecd00abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:30.798465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:30.872991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:31.027734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78599fb-f383-4a4b-b541-9518a0d20795 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:31.030091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c78599fb-f383-4a4b-b541-9518a0d20795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:31.208229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78599fb-f383-4a4b-b541-9518a0d20795 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:31.254593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:31.340479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:31.341134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:31.341412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:31.341684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:31.343410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:31.444149 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:31.445885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11988fc0-cdcd-4f44-b84a-86a8fb63ef7d service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:31.465168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 700/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:32.363496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3808dd-b006-40dd-9606-77727da2e2aa None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:32.364413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3808dd-b006-40dd-9606-77727da2e2aa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:32.364413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3808dd-b006-40dd-9606-77727da2e2aa None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:32.366752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:32.375310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0a39-dd7f-41c2-8d11-3235d524a685 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:32.376031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0a39-dd7f-41c2-8d11-3235d524a685 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:32.376183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a0a39-dd7f-41c2-8d11-3235d524a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:32.435102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:32.460463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4e9a0a39-dd7f-41c2-8d11-3235d524a685 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:32.461722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4e9a0a39-dd7f-41c2-8d11-3235d524a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:32.569054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:32.730916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3808dd-b006-40dd-9606-77727da2e2aa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:32.733395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8e3808dd-b006-40dd-9606-77727da2e2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:32.894226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3808dd-b006-40dd-9606-77727da2e2aa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:32.908545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:33.018640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:33.019211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:33.019486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:33.019765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:33.021453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:33.116053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:33.117686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d0546-7550-41da-b5cf-46f2bdeb4309 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:33.137937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 721/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:34.010213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1995d-bed1-4d42-824c-b7c1149f626e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:34.010956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1995d-bed1-4d42-824c-b7c1149f626e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:34.010956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1995d-bed1-4d42-824c-b7c1149f626e None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:34.013258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:34.019305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265108-da7c-4524-b4d6-1713a57662d5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:34.019741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265108-da7c-4524-b4d6-1713a57662d5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:34.019877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75265108-da7c-4524-b4d6-1713a57662d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:34.040158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:34.102246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-75265108-da7c-4524-b4d6-1713a57662d5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:34.103496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-75265108-da7c-4524-b4d6-1713a57662d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:34.202866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:34.353034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1995d-bed1-4d42-824c-b7c1149f626e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:34.354677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ebd1995d-bed1-4d42-824c-b7c1149f626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:34.489957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:34.532118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1995d-bed1-4d42-824c-b7c1149f626e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:34.655147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:34.655827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:34.656111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:34.656371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:34.658012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:34.738204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:34.757736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:34.759915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a5a7f-e9f3-4303-9614-e543f9cabe33 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:34.780959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 703/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:35.652978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7c5c-d1f1-4b7c-a44b-9a5bd4ff02dc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:35.653839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7c5c-d1f1-4b7c-a44b-9a5bd4ff02dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:35.653839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7c5c-d1f1-4b7c-a44b-9a5bd4ff02dc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:35.656223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:35.661437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62870f6-b0b9-4e8d-98cc-87bf1ef9b77b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:35.661894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62870f6-b0b9-4e8d-98cc-87bf1ef9b77b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:35.661989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62870f6-b0b9-4e8d-98cc-87bf1ef9b77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:35.740065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e62870f6-b0b9-4e8d-98cc-87bf1ef9b77b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:35.741917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e62870f6-b0b9-4e8d-98cc-87bf1ef9b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:35.965217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:36.115963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7c5c-d1f1-4b7c-a44b-9a5bd4ff02dc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:36.118243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-47fb7c5c-d1f1-4b7c-a44b-9a5bd4ff02dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:36.217077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:36.295858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7c5c-d1f1-4b7c-a44b-9a5bd4ff02dc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:36.424049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:36.424708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:36.424991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:36.425263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:36.427046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:36.477783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:36.522001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:36.523667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a43d8-9399-44d6-9c21-0d84183ffb2a service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:36.543025 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 724/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:37.369294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d0e8-c47c-4d42-af01-7cb6a6ec16d5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:37.369904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d0e8-c47c-4d42-af01-7cb6a6ec16d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:37.369904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58d0e8-c47c-4d42-af01-7cb6a6ec16d5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:37.371249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:37.377650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c925a3-242d-4b1f-9fbb-bf4968ffd2c6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:37.378403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c925a3-242d-4b1f-9fbb-bf4968ffd2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:37.378596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c925a3-242d-4b1f-9fbb-bf4968ffd2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:37.463946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a1c925a3-242d-4b1f-9fbb-bf4968ffd2c6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:37.465721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a1c925a3-242d-4b1f-9fbb-bf4968ffd2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:37.573808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:37.727712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58d0e8-c47c-4d42-af01-7cb6a6ec16d5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:37.729239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bb58d0e8-c47c-4d42-af01-7cb6a6ec16d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:37.782310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:37.886666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58d0e8-c47c-4d42-af01-7cb6a6ec16d5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:38.022334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:38.023117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:38.023397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:38.023671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:38.025320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:38.032611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:38.116945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:38.118573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65b1f5-6567-4306-b6dd-5285eedca400 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:38.137268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 706/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:38.952875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7daf3e-033f-4b03-a43d-1b63fcdda38d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:38.953418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7daf3e-033f-4b03-a43d-1b63fcdda38d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:38.953418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7daf3e-033f-4b03-a43d-1b63fcdda38d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:38.954866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:38.961371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a6aac-4136-47c7-8e9d-93b1121600cf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:38.961842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a6aac-4136-47c7-8e9d-93b1121600cf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:38.961942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a6aac-4136-47c7-8e9d-93b1121600cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:39.056053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-df7a6aac-4136-47c7-8e9d-93b1121600cf None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:39.057203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-df7a6aac-4136-47c7-8e9d-93b1121600cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:39.137234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:39.177843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:39.334003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7daf3e-033f-4b03-a43d-1b63fcdda38d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:39.336374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5a7daf3e-033f-4b03-a43d-1b63fcdda38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:39.361439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:39.501203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7daf3e-033f-4b03-a43d-1b63fcdda38d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:39.605934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:39.635029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:39.635701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:39.635953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:39.636188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:39.637875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:39.739657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:39.741478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35752463-9b4c-42fa-9ab7-30a4c0e13018 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:39.760892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 727/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:40.583012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573bd2-6b5b-44a4-8542-a692ed6e19db None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:40.583762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573bd2-6b5b-44a4-8542-a692ed6e19db None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:40.583762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea573bd2-6b5b-44a4-8542-a692ed6e19db None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:40.584998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:40.591125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8f19-1bbd-49ac-8da5-19eb178565e3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:40.591911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8f19-1bbd-49ac-8da5-19eb178565e3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:40.592066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac8f19-1bbd-49ac-8da5-19eb178565e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:40.678548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a8ac8f19-1bbd-49ac-8da5-19eb178565e3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:40.679861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a8ac8f19-1bbd-49ac-8da5-19eb178565e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:40.730617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:40.790537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:40.942491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573bd2-6b5b-44a4-8542-a692ed6e19db None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:40.944874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ea573bd2-6b5b-44a4-8542-a692ed6e19db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:41.117037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea573bd2-6b5b-44a4-8542-a692ed6e19db None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:41.177677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:41.238685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:41.239427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:41.239858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:41.240241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:41.241973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:41.333687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:41.335373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb865e7-6ed7-4f84-9c25-88ba7f250b79 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:41.353600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 709/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:42.177480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ce450-841a-42e4-af8e-3022ee4a1e69 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:42.178100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ce450-841a-42e4-af8e-3022ee4a1e69 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:42.178100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ce450-841a-42e4-af8e-3022ee4a1e69 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:42.179454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:42.187357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23773065-50fb-42f5-80ac-2dc1d412acb9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:42.187849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23773065-50fb-42f5-80ac-2dc1d412acb9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:42.187849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23773065-50fb-42f5-80ac-2dc1d412acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:42.274198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-23773065-50fb-42f5-80ac-2dc1d412acb9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:42.275459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-23773065-50fb-42f5-80ac-2dc1d412acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:42.277609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:42.374309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:19:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:42.522982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ce450-841a-42e4-af8e-3022ee4a1e69 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:42.524568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-215ce450-841a-42e4-af8e-3022ee4a1e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:42.683734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ce450-841a-42e4-af8e-3022ee4a1e69 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:42.711852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:42.806459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:42.807045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:42.807331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:42.807627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:42.809277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:42.906577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:42.908234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea62c0-361e-4c95-884d-10f02b47d4da service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:42.927284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 730/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:44.829389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006e59-1e6a-431d-91b7-369a06df7947 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:44.829997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006e59-1e6a-431d-91b7-369a06df7947 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:44.829997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3006e59-1e6a-431d-91b7-369a06df7947 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:44.831410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:44.837782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f865c5-df68-4228-b96b-4868a6fa15ca None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:44.838424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f865c5-df68-4228-b96b-4868a6fa15ca None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:44.838624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f865c5-df68-4228-b96b-4868a6fa15ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:44.886740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:44.916585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e4f865c5-df68-4228-b96b-4868a6fa15ca None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:44.917803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e4f865c5-df68-4228-b96b-4868a6fa15ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:45.020312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:45.059014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006e59-1e6a-431d-91b7-369a06df7947 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:45.060602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d3006e59-1e6a-431d-91b7-369a06df7947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:45.177317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3006e59-1e6a-431d-91b7-369a06df7947 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:45.212277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:45.267224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:45.267888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:45.268171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:45.268452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:45.270575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:45.397056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:45.398549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ac930-2d74-4068-af06-a0e8dd7d9e12 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:45.426135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 712/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:47.729385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c72c5e-8acb-441a-8a74-ea9171406c31 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:47.730017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c72c5e-8acb-441a-8a74-ea9171406c31 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:47.730017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c72c5e-8acb-441a-8a74-ea9171406c31 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:47.731413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:47.739795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f9f5c-2419-482b-be37-df094ff31f2a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:47.740233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f9f5c-2419-482b-be37-df094ff31f2a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:47.740333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f9f5c-2419-482b-be37-df094ff31f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:47.810482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-187f9f5c-2419-482b-be37-df094ff31f2a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:47.811678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-187f9f5c-2419-482b-be37-df094ff31f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:47.826181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:47.931007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:47.973592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c72c5e-8acb-441a-8a74-ea9171406c31 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:47.975195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-57c72c5e-8acb-441a-8a74-ea9171406c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:48.087049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c72c5e-8acb-441a-8a74-ea9171406c31 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:48.100076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:48.163656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:48.164583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:48.164867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:48.165142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:48.166858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:48.290878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:48.292364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd2058-c36b-4cc5-9dbf-e884c450e9cc service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:48.311808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 733/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:50.644869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1c44-34ed-455b-ae19-c065110483de None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:50.645505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1c44-34ed-455b-ae19-c065110483de None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:50.645505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e1c44-34ed-455b-ae19-c065110483de None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:50.646851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:50.655923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af71e4-873d-430f-b51c-76c4fe88e0ca None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:50.656703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af71e4-873d-430f-b51c-76c4fe88e0ca None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:50.656818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af71e4-873d-430f-b51c-76c4fe88e0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:50.715407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:50.735741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-33af71e4-873d-430f-b51c-76c4fe88e0ca None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:50.736966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-33af71e4-873d-430f-b51c-76c4fe88e0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:50.851294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:50.893356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e1c44-34ed-455b-ae19-c065110483de None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:50.895451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7b1e1c44-34ed-455b-ae19-c065110483de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:51.012143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e1c44-34ed-455b-ae19-c065110483de None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:51.063021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:51.103442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:51.104096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:51.104378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:51.104649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:51.106289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:51.226434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:51.227916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f1b45-7390-4c10-b313-0eb27c8ec4b2 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:51.247491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 715/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:19:53.381648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca45cc-9313-4f1f-a966-b9703f660fb6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:53.382262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca45cc-9313-4f1f-a966-b9703f660fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:53.382262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca45cc-9313-4f1f-a966-b9703f660fb6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:53.383641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:53.389938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74dd7c-205d-4ec8-8116-e0dc3e6e7d4b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:53.390424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74dd7c-205d-4ec8-8116-e0dc3e6e7d4b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:53.390546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74dd7c-205d-4ec8-8116-e0dc3e6e7d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:53.465579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ea74dd7c-205d-4ec8-8116-e0dc3e6e7d4b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:53.466816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ea74dd7c-205d-4ec8-8116-e0dc3e6e7d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:53.489799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:53.572734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:53.612367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca45cc-9313-4f1f-a966-b9703f660fb6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:53.613917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04ca45cc-9313-4f1f-a966-b9703f660fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:53.729173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca45cc-9313-4f1f-a966-b9703f660fb6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:53.749677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:53.806554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:53.807208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:53.807489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:53.807767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:53.809434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:53.933041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:53.934574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baf1c3-6abe-4c33-a829-9896c5acbefb service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:53.955087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 736/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:19:56.051134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c6a3-9004-4ead-8259-1e7a73051c11 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:56.051692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c6a3-9004-4ead-8259-1e7a73051c11 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:56.051692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c6a3-9004-4ead-8259-1e7a73051c11 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:56.053771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:56.062831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e74a87-2af1-496c-95fb-926ddeff2bb0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:56.063288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e74a87-2af1-496c-95fb-926ddeff2bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:56.063505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e74a87-2af1-496c-95fb-926ddeff2bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:56.134822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:56.137781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c1e74a87-2af1-496c-95fb-926ddeff2bb0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:56.139617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1e74a87-2af1-496c-95fb-926ddeff2bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:56.250202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:56.290313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c6a3-9004-4ead-8259-1e7a73051c11 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:56.291919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1690c6a3-9004-4ead-8259-1e7a73051c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:56.409165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c6a3-9004-4ead-8259-1e7a73051c11 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:56.471523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:56.499095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:56.499756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:56.500042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:56.500385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:56.502028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:56.615890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:56.617341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750dd8-e120-47b4-94ec-a1bf845f3083 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:56.636660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 718/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:19:58.725879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b34a4-e593-4b0e-9b42-b01dde1b1bd2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:58.726848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b34a4-e593-4b0e-9b42-b01dde1b1bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:58.726848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b34a4-e593-4b0e-9b42-b01dde1b1bd2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:58.729218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:19:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:19:58.734942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60853e5a-851a-49a9-9dbf-c942224f49f1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:58.735390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60853e5a-851a-49a9-9dbf-c942224f49f1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:58.735496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60853e5a-851a-49a9-9dbf-c942224f49f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:58.801464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-60853e5a-851a-49a9-9dbf-c942224f49f1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:19:58.802725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-60853e5a-851a-49a9-9dbf-c942224f49f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:58.834476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:58.905034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:19:58.943385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b34a4-e593-4b0e-9b42-b01dde1b1bd2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:19:58.945316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-475b34a4-e593-4b0e-9b42-b01dde1b1bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:59.065101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b34a4-e593-4b0e-9b42-b01dde1b1bd2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:19:59.096732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:19:59.145627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:19:59.146283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:19:59.146678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:19:59.146894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:19:59.148518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:19:59.279737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:19:59.281220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd0ed2-83b2-49db-9963-b536b8907695 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:19:59.301226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 739/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:01.417759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef890c-93c6-4d50-a7e3-67ffc14af972 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:01.418324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef890c-93c6-4d50-a7e3-67ffc14af972 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:01.418324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef890c-93c6-4d50-a7e3-67ffc14af972 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:01.420516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:20:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:01.430875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8984ab4-46b7-43cd-a7c8-6cc3b37b1ab9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:01.431353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8984ab4-46b7-43cd-a7c8-6cc3b37b1ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:01.431457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8984ab4-46b7-43cd-a7c8-6cc3b37b1ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:01.512074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f8984ab4-46b7-43cd-a7c8-6cc3b37b1ab9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:01.513674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f8984ab4-46b7-43cd-a7c8-6cc3b37b1ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:01.526548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:01.632340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:01.671587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef890c-93c6-4d50-a7e3-67ffc14af972 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:01.673076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5fef890c-93c6-4d50-a7e3-67ffc14af972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:01.788356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef890c-93c6-4d50-a7e3-67ffc14af972 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:01.853333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:01.875644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:01.876296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:01.876615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:01.876910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:01.878594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:02.000436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:02.002025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3238cb40-5137-412e-a3e9-248a5d0fec54 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:02.023022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 721/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:04.140321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124eb2-97de-47d5-b451-1898732a7eef None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:04.140912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124eb2-97de-47d5-b451-1898732a7eef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:04.140912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52124eb2-97de-47d5-b451-1898732a7eef None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:04.142392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:04.149814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b311ae0-b2f3-4f5c-81bd-e25ccf03c9a1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:04.150312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b311ae0-b2f3-4f5c-81bd-e25ccf03c9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:04.150418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b311ae0-b2f3-4f5c-81bd-e25ccf03c9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:04.237622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0b311ae0-b2f3-4f5c-81bd-e25ccf03c9a1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:04.238910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b311ae0-b2f3-4f5c-81bd-e25ccf03c9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:04.280940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:04.356071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:04.396653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124eb2-97de-47d5-b451-1898732a7eef None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:04.399064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-52124eb2-97de-47d5-b451-1898732a7eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:04.516016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52124eb2-97de-47d5-b451-1898732a7eef None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:04.558547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:04.594835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:04.595569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:04.595852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:04.596127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:04.597787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:04.718403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:04.720211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010bbfb4-483d-4efc-a646-c5e17ba98c8d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:04.739904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 742/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:06.852005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceca3f-7ab2-4bc3-8664-e5e112d67708 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:06.852611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceca3f-7ab2-4bc3-8664-e5e112d67708 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:06.852611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ceca3f-7ab2-4bc3-8664-e5e112d67708 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:06.853923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:06.859723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453895-18fc-4cf4-99c4-651f686ec6e4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:06.860366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453895-18fc-4cf4-99c4-651f686ec6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:06.860366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8453895-18fc-4cf4-99c4-651f686ec6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:06.943537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d8453895-18fc-4cf4-99c4-651f686ec6e4 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:06.944840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d8453895-18fc-4cf4-99c4-651f686ec6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:06.964797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:07.060977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:07.099174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceca3f-7ab2-4bc3-8664-e5e112d67708 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:07.100769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69ceca3f-7ab2-4bc3-8664-e5e112d67708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:07.231673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ceca3f-7ab2-4bc3-8664-e5e112d67708 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:07.317020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:07.331135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:07.332054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:07.332377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:07.332656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:07.335282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:07.466905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:07.468355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41833cb-92b6-4e6f-922d-9ac7d40a81d1 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:07.493034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 724/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:09.512558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8519-ca7d-4657-b988-257c898e6813 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:09.513464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8519-ca7d-4657-b988-257c898e6813 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:09.513464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8519-ca7d-4657-b988-257c898e6813 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:09.515884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:20:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:09.521508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d2b28-d721-4997-81f5-fde399fe7327 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:09.521967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d2b28-d721-4997-81f5-fde399fe7327 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:09.522085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d2b28-d721-4997-81f5-fde399fe7327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:09.590972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5e4d2b28-d721-4997-81f5-fde399fe7327 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:09.594167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5e4d2b28-d721-4997-81f5-fde399fe7327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:09.653372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:09.711857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:09.752517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b8519-ca7d-4657-b988-257c898e6813 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:09.754992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1c5b8519-ca7d-4657-b988-257c898e6813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:09.889588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b8519-ca7d-4657-b988-257c898e6813 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:09.951417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:09.982275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:09.982989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:09.983233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:09.983521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:09.985579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:10.105161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:10.106906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7590b-00e1-46b8-a0d8-9ded604b3c8d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:10.126556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 745/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:19.709682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=22 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:20.123046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=66 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:29.652221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8420a1-6f97-4ac4-8ad1-d9fee989cf06 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:29.653139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8420a1-6f97-4ac4-8ad1-d9fee989cf06 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:29.653139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8420a1-6f97-4ac4-8ad1-d9fee989cf06 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:29.656166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 726/1471] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:29.665113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f57f5f-6924-4856-819a-c22ca73e5f80 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:29.665576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f57f5f-6924-4856-819a-c22ca73e5f80 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:29.665684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f57f5f-6924-4856-819a-c22ca73e5f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:29.863438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-59f57f5f-6924-4856-819a-c22ca73e5f80 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:29.864697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-59f57f5f-6924-4856-819a-c22ca73e5f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:29.975515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 746/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:30.024967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6258be3-8ea5-46bc-8dfd-0329ffe894e2 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:30.025260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6258be3-8ea5-46bc-8dfd-0329ffe894e2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:30.025536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6258be3-8ea5-46bc-8dfd-0329ffe894e2 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:30.026946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 727/1473] 10.222.0.40 () {54 vars in 960 bytes} [Fri Jul 15 15:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:30.033453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5331-ff3a-4568-a5dc-569d9aecb97f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:30.033805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5331-ff3a-4568-a5dc-569d9aecb97f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:30.034207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf5331-ff3a-4568-a5dc-569d9aecb97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:30.126688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2bcf5331-ff3a-4568-a5dc-569d9aecb97f None None] MFA Rules not processed for user `9fe7e2ad26c8483e8aecb89225f3b26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:30.127920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2bcf5331-ff3a-4568-a5dc-569d9aecb97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:30.149014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:30.216626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 747/1474] 10.222.0.40 () {58 vars in 1057 bytes} [Fri Jul 15 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:30.227746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6258be3-8ea5-46bc-8dfd-0329ffe894e2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:30.229859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6258be3-8ea5-46bc-8dfd-0329ffe894e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:30.345828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6258be3-8ea5-46bc-8dfd-0329ffe894e2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:30.423976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:30.424638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:30.424914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:30.425190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:30.426878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:30.486575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:30.547887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:30.549329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40b05-e27c-4bca-ac84-6e089952965f service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:30.575945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 728/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:32.732081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a814c2-d7eb-4147-a5ba-941b3fbe020a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:32.732713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a814c2-d7eb-4147-a5ba-941b3fbe020a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:32.732713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a814c2-d7eb-4147-a5ba-941b3fbe020a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:32.734030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 748/1476] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:32.740488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b3878-faaf-4e10-ae8f-217b2c85699e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:32.741159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b3878-faaf-4e10-ae8f-217b2c85699e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:32.741314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b3878-faaf-4e10-ae8f-217b2c85699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:32.832433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-be0b3878-faaf-4e10-ae8f-217b2c85699e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:32.833669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be0b3878-faaf-4e10-ae8f-217b2c85699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:32.942165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 729/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:32.973853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a814c2-d7eb-4147-a5ba-941b3fbe020a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:32.975490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-19a814c2-d7eb-4147-a5ba-941b3fbe020a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:33.126326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a814c2-d7eb-4147-a5ba-941b3fbe020a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:33.176467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:33.223526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:33.224215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:33.224487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:33.224764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:33.225495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:33.227335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:20:33.227335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:20:33.227335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:20:33.228483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2142c6-5ec1-44c2-b527-de2ac179c094 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:33.249003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 749/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Fri Jul 15 15:20:32 2022] GET /identity/v3/projects => generated 2553 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:20:35.302928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adfeac-5cc6-453d-992f-679464def0f9 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:35.303850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adfeac-5cc6-453d-992f-679464def0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:35.303850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adfeac-5cc6-453d-992f-679464def0f9 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:35.305949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 730/1479] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:35.312917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb069cfb-21c1-4e42-af0d-45e7e069b7a5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:35.313392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb069cfb-21c1-4e42-af0d-45e7e069b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:35.313503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb069cfb-21c1-4e42-af0d-45e7e069b7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:35.391541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eb069cfb-21c1-4e42-af0d-45e7e069b7a5 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:35.392805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eb069cfb-21c1-4e42-af0d-45e7e069b7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:35.494320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 750/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:35.546079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adfeac-5cc6-453d-992f-679464def0f9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:35.547726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f2adfeac-5cc6-453d-992f-679464def0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:35.558025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:35.703501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adfeac-5cc6-453d-992f-679464def0f9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:35.784378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:35.820172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:35.820756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:35.821034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:35.821312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:35.823017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:35.920415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:35.922028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8160c-602e-449f-84ec-28f0a06439c2 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:35.941007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 731/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:38.177111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6bcf-bd29-4074-a4fe-8359b77da577 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:38.178770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6bcf-bd29-4074-a4fe-8359b77da577 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:38.178770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d6bcf-bd29-4074-a4fe-8359b77da577 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:38.180034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 751/1482] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:38.187601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4b7e8-3351-43c2-8d5e-ae39b9a38a64 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:38.188118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4b7e8-3351-43c2-8d5e-ae39b9a38a64 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:38.188118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4b7e8-3351-43c2-8d5e-ae39b9a38a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:38.265263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-13f4b7e8-3351-43c2-8d5e-ae39b9a38a64 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:38.266475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13f4b7e8-3351-43c2-8d5e-ae39b9a38a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:38.313433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:38.373847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 732/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:38.411678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6bcf-bd29-4074-a4fe-8359b77da577 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:38.414002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c93d6bcf-bd29-4074-a4fe-8359b77da577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:38.443962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:38.549801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d6bcf-bd29-4074-a4fe-8359b77da577 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:38.626876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:38.627527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:38.627807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:38.628081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:38.629747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:38.721143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:38.760685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:38.762501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c5d2e-93ce-4e7b-9b1b-e1a0cbdf3efe service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:38.784539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 752/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:40.868911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641472b-fc42-48ec-abb4-51e3f1e004f3 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:40.869767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641472b-fc42-48ec-abb4-51e3f1e004f3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:40.869767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641472b-fc42-48ec-abb4-51e3f1e004f3 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:40.871828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 733/1485] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:40.878425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada558-50c2-41f8-9cff-39224dd19ca1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:40.878905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada558-50c2-41f8-9cff-39224dd19ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:40.879053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fada558-50c2-41f8-9cff-39224dd19ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:40.952904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9fada558-50c2-41f8-9cff-39224dd19ca1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:40.954146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fada558-50c2-41f8-9cff-39224dd19ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:41.057110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 753/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:41.109929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641472b-fc42-48ec-abb4-51e3f1e004f3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:41.111541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8641472b-fc42-48ec-abb4-51e3f1e004f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:41.229647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641472b-fc42-48ec-abb4-51e3f1e004f3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:41.313219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:41.315221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:41.315985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:41.316883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:41.317180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:41.319425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:41.443181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:41.444635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f4f98-c06b-4df8-ad50-067288af5e91 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:41.465122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 734/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:43.583427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151822e3-53ee-484e-adf2-828a433d09d5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:43.584038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151822e3-53ee-484e-adf2-828a433d09d5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:43.584038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151822e3-53ee-484e-adf2-828a433d09d5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:43.586093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 754/1488] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:43.593713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f262-5a26-4f1d-b55b-ed52583e28fa None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:43.594181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f262-5a26-4f1d-b55b-ed52583e28fa None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:43.594288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153f262-5a26-4f1d-b55b-ed52583e28fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:43.675020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b153f262-5a26-4f1d-b55b-ed52583e28fa None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:43.676239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b153f262-5a26-4f1d-b55b-ed52583e28fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:43.748300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:43.804516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 735/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:43.845310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151822e3-53ee-484e-adf2-828a433d09d5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:43.846980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-151822e3-53ee-484e-adf2-828a433d09d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:43.887718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:43.972346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151822e3-53ee-484e-adf2-828a433d09d5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:44.048322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:44.048977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:44.049259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:44.049535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:44.051229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:44.140815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:44.171328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:44.172793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce94ccb-f82b-460d-8221-957801062206 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:44.199443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 755/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:46.435867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10260176-39e3-4c66-9daa-ac95bfabb338 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:46.436790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10260176-39e3-4c66-9daa-ac95bfabb338 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:46.436790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10260176-39e3-4c66-9daa-ac95bfabb338 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:46.438965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 736/1491] 10.222.0.40 () {54 vars in 926 bytes} [Fri Jul 15 15:20:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:20:46.444908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187230-3c6e-459d-99c7-b9e6165a3c3c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:46.445358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187230-3c6e-459d-99c7-b9e6165a3c3c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:46.445455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29187230-3c6e-459d-99c7-b9e6165a3c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:46.519144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-29187230-3c6e-459d-99c7-b9e6165a3c3c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:20:46.520382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-29187230-3c6e-459d-99c7-b9e6165a3c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:46.623336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 756/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Fri Jul 15 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:20:46.666705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10260176-39e3-4c66-9daa-ac95bfabb338 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:20:46.668783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-10260176-39e3-4c66-9daa-ac95bfabb338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:46.784826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10260176-39e3-4c66-9daa-ac95bfabb338 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:20:46.859088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:20:46.859719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:20:46.860023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:20:46.860281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:20:46.861938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:20:46.871305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:46.987673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:20:46.989119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c92fbe-2be4-42d1-82e1-087d019b83a4 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:20:47.008200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 737/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:20:56.621423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=82 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:20:57.005710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=50 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:14.662422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:14.663075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:14.663075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:14.741662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:22:14.742954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:14.859536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 757/1494] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:22:14.871377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:14.871895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:14.871895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:14.951082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:22:14.952305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:15.055664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 738/1495] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:22:15.063416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:15.064864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:15.076485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4004d-b477-4c8e-bd22-442c6c6c682c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:15.146354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:15.190461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:15.191158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:15.191438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:15.191711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:15.192445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:22:15.192753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:15.193840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:15.193840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:22:15.193840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:22:15.194692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:15.207428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 758/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:22:15.216615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:15.227695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3536-d1aa-4fa0-85d4-933d38399dc2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:15.303618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:15.342791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:15.343477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:15.343739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:15.344121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:15.344813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:15.346822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:15.346822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:22:15.346822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:22:15.348021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:15.412229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 739/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:22:15 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:22:15.416691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:15.418215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:15.456188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec204d5-1446-4f6a-bd88-b3297c184848 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:15.566686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:15.567356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:15.567632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:15.567888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:15.568594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:15.569632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:15.569632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:22:15.569632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:22:15.570861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:15.613715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 759/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:22:15 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:22:15.620044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:15.622363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:15.693578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7564-14fb-414d-bfe3-5903959f518a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:15.786491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:15.803170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:15.804193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:15.804480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:15.804754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:15.805477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:15.806617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:15.806617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:22:15.806617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:22:15.807468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:15.820816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 740/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:22:15.826451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:15.828010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:15.839846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b91ac-c359-4d44-a239-6c55380a57b7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:15.868282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:15.954685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:15.955376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:15.955674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:15.955929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:15.956670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:15.957711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:15.957711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:22:15.957711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:22:15.958652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0f475-a967-49b9-922f-bd5c0b7077fc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:15.972672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 760/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:22:15.977879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:15.980286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:16.022150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de15f6c-4c79-4d80-bdf1-be2f1da192c3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:16.126400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:16.127529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:16.127818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:16.128098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] PATH_INFO: `/v3/projects/d65f3654502a45c99b08941a85172476/users/fa9172a264804f8f9e2354cd5be08f45/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:16.152595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65f3654502a45c99b08941a85172476, user_id=fa9172a264804f8f9e2354cd5be08f45, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:16.153633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:16.153633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:22:16.153633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:22:16.155756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6e693-720e-4a46-afc5-0c383b4450b9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:16.187413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 741/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:22:15 2022] PUT /identity/v3/projects/d65f3654502a45c99b08941a85172476/users/fa9172a264804f8f9e2354cd5be08f45/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:22:16.200021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc45e0-b760-4129-a18c-c9e33a96b568 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:16.200313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc45e0-b760-4129-a18c-c9e33a96b568 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:16.200593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc45e0-b760-4129-a18c-c9e33a96b568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:16.260091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-18fc45e0-b760-4129-a18c-c9e33a96b568 None None] MFA Rules not processed for user `fa9172a264804f8f9e2354cd5be08f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:22:16.261315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-18fc45e0-b760-4129-a18c-c9e33a96b568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:16.343331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 761/1502] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:22:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:22:16.356858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c5c0-380f-4e1e-bab5-a46d3cf8355f None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:16.357173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c5c0-380f-4e1e-bab5-a46d3cf8355f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:16.357451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c5c0-380f-4e1e-bab5-a46d3cf8355f None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:16.358902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 742/1503] 10.222.0.40 () {54 vars in 963 bytes} [Fri Jul 15 15:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:22:16.363996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66020135-0184-4216-9084-db7d3520bfee None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:16.364286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66020135-0184-4216-9084-db7d3520bfee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:16.364564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66020135-0184-4216-9084-db7d3520bfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:16.396538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:16.445677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-66020135-0184-4216-9084-db7d3520bfee None None] MFA Rules not processed for user `4cfd0067b59048f1850e8eb6e94640ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:22:16.446962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-66020135-0184-4216-9084-db7d3520bfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:16.527930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 762/1504] 10.222.0.40 () {58 vars in 1060 bytes} [Fri Jul 15 15:22:16 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:22:16.536576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c5c0-380f-4e1e-bab5-a46d3cf8355f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:16.538994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0f12c5c0-380f-4e1e-bab5-a46d3cf8355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:16.599545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:16.667780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c5c0-380f-4e1e-bab5-a46d3cf8355f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:16.747730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:16.748382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:16.748664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:16.748939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:16.750636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:16.864208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:16.865564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:16.869173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:16.886614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 743/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:22:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:22:16.931649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78930-4e6a-4348-96f8-6d098894a0f5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:16.932090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78930-4e6a-4348-96f8-6d098894a0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:16.932090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a78930-4e6a-4348-96f8-6d098894a0f5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:16.933423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 763/1506] 10.222.0.40 () {52 vars in 829 bytes} [Fri Jul 15 15:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:22:16.937726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:16.939308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:17.043550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b535a0e-2422-4d93-b174-9c288b76bcf9 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:17.151052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:17.151837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:17.152111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:17.152383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] PATH_INFO: `/v3/users/fa9172a264804f8f9e2354cd5be08f45` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:17.165370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9172a264804f8f9e2354cd5be08f45)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:17.166401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:17.166401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:22:17.166401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:22:17.167651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:17.177824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:17.236620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9172a264804f8f9e2354cd5be08f45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:17.247302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9172a264804f8f9e2354cd5be08f45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:17.247896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9172a264804f8f9e2354cd5be08f45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:17.248402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9172a264804f8f9e2354cd5be08f45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:17.261152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9172a264804f8f9e2354cd5be08f45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:17.287514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 744/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:22:16 2022] DELETE /identity/v3/users/fa9172a264804f8f9e2354cd5be08f45 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:22:17.292756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:17.293204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:17.293601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:17.368711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:22:17.369947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:17.453009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:17.466799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 764/1508] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:22:17.478025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:17.479976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:17.600096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1ce5f-72dc-4d36-92b6-4d3335a5ac9b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:17.661763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:17.687779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:17.688790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:17.689184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:17.689469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:17.691137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:17.804814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:17.806183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8995ba2-93c5-45f2-9359-a473c59464ba service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:17.824987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 745/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:22:18.261950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:22:18.263277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:22:18.275084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f78e480-b12c-49ce-bc80-4cbf15ea7202 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:22:18.384116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:22:18.385220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:22:18.385504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:22:18.385780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] PATH_INFO: `/v3/projects/d65f3654502a45c99b08941a85172476` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:22:18.396655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65f3654502a45c99b08941a85172476)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:22:18.397714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:22:18.397714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:22:18.397714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:22:18.398970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:22:18.471384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65f3654502a45c99b08941a85172476', 'request_id': 'req-062c4b30-fee1-4b63-8abc-24479b442657'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:18.481259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65f3654502a45c99b08941a85172476', 'request_id': 'req-062c4b30-fee1-4b63-8abc-24479b442657'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:18.497138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65f3654502a45c99b08941a85172476', 'request_id': 'req-062c4b30-fee1-4b63-8abc-24479b442657'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:18.521098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] The token cache is being invalidate because project d65f3654502a45c99b08941a85172476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:22:18.521385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:18.521807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-062c4b30-fee1-4b63-8abc-24479b442657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:22:18.523290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 765/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:22:18 2022] DELETE /identity/v3/projects/d65f3654502a45c99b08941a85172476 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:22:27.822556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=62 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:22:28.519043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:24:07.500068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65987c6-df7a-416e-9fc5-c16167d60f2a None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:07.501097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65987c6-df7a-416e-9fc5-c16167d60f2a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:07.501097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65987c6-df7a-416e-9fc5-c16167d60f2a None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:07.503455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:24:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:07.508664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc248aae-f1b8-4073-b3ec-be9d59c69b81 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:07.509150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc248aae-f1b8-4073-b3ec-be9d59c69b81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:07.509267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc248aae-f1b8-4073-b3ec-be9d59c69b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:07.595473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fc248aae-f1b8-4073-b3ec-be9d59c69b81 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:07.596864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc248aae-f1b8-4073-b3ec-be9d59c69b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:07.715008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:07.882479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65987c6-df7a-416e-9fc5-c16167d60f2a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:07.885542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d65987c6-df7a-416e-9fc5-c16167d60f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:08.012917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65987c6-df7a-416e-9fc5-c16167d60f2a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:08.018915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74dd14-9ffe-49b5-b3c6-b9dd3f74547a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:08.019605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74dd14-9ffe-49b5-b3c6-b9dd3f74547a service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:08.019900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74dd14-9ffe-49b5-b3c6-b9dd3f74547a service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:08.020169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74dd14-9ffe-49b5-b3c6-b9dd3f74547a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:08.028798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74dd14-9ffe-49b5-b3c6-b9dd3f74547a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:08.030440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74dd14-9ffe-49b5-b3c6-b9dd3f74547a service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:08.040474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 747/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:08.880859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a05b-9b39-4d7a-b25a-504ef867a9fc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:08.881443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a05b-9b39-4d7a-b25a-504ef867a9fc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:08.881443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360a05b-9b39-4d7a-b25a-504ef867a9fc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:08.882876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:08.888707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370be049-83b1-4edd-9043-a4acb8288efb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:08.889239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370be049-83b1-4edd-9043-a4acb8288efb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:08.889453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370be049-83b1-4edd-9043-a4acb8288efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:08.911084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-370be049-83b1-4edd-9043-a4acb8288efb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:08.912419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-370be049-83b1-4edd-9043-a4acb8288efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:08.922506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:09.084220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360a05b-9b39-4d7a-b25a-504ef867a9fc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:09.110321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360a05b-9b39-4d7a-b25a-504ef867a9fc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:09.117628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b018f-a28e-4cf8-81f7-85232550ad77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:09.118295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b018f-a28e-4cf8-81f7-85232550ad77 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:09.118560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b018f-a28e-4cf8-81f7-85232550ad77 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:09.118883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b018f-a28e-4cf8-81f7-85232550ad77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:09.128854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b018f-a28e-4cf8-81f7-85232550ad77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:09.130497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b018f-a28e-4cf8-81f7-85232550ad77 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:09.141147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 768/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:24:10.587696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44190e9-92f7-44dd-a67e-733ab2a2622e None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:10.588331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44190e9-92f7-44dd-a67e-733ab2a2622e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:10.588331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44190e9-92f7-44dd-a67e-733ab2a2622e None None] PATH_INFO: `/v3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:10.589834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 749/1517] 10.222.0.27 () {54 vars in 925 bytes} [Fri Jul 15 15:24:10 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:24:10.596400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f78368-48df-4625-9f53-a72130d9c902 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:10.596840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f78368-48df-4625-9f53-a72130d9c902 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:10.596974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f78368-48df-4625-9f53-a72130d9c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:10.662480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-70f78368-48df-4625-9f53-a72130d9c902 None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:10.663751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-70f78368-48df-4625-9f53-a72130d9c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:10.684172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:24:10.749665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 769/1518] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Jul 15 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:15.007425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04b1c-31a4-4c7d-a3d8-088b47053c72 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:15.007995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04b1c-31a4-4c7d-a3d8-088b47053c72 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:15.007995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04b1c-31a4-4c7d-a3d8-088b47053c72 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:15.010029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 750/1519] 10.222.0.40 () {54 vars in 960 bytes} [Fri Jul 15 15:24:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:15.017347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa209da7-c6cc-4734-943e-63f878597c5c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:15.017870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa209da7-c6cc-4734-943e-63f878597c5c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:15.017870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa209da7-c6cc-4734-943e-63f878597c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:15.037380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-aa209da7-c6cc-4734-943e-63f878597c5c None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:15.038562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aa209da7-c6cc-4734-943e-63f878597c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:15.048637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 770/1520] 10.222.0.40 () {58 vars in 1057 bytes} [Fri Jul 15 15:24:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:15.056393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04b1c-31a4-4c7d-a3d8-088b47053c72 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:15.071617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04b1c-31a4-4c7d-a3d8-088b47053c72 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:15.077124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9f428-25a7-48f6-8624-1b4f1cbfaace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:15.077757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9f428-25a7-48f6-8624-1b4f1cbfaace service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:15.078034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9f428-25a7-48f6-8624-1b4f1cbfaace service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:15.078305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9f428-25a7-48f6-8624-1b4f1cbfaace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:15.087478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9f428-25a7-48f6-8624-1b4f1cbfaace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:15.089055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9f428-25a7-48f6-8624-1b4f1cbfaace service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:15.099820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 751/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:20.413119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19143c-ffb6-439a-a7fc-79953ef01582 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:20.414097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19143c-ffb6-439a-a7fc-79953ef01582 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:20.414097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19143c-ffb6-439a-a7fc-79953ef01582 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:20.416321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:24:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:20.423265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ecb2e-6564-4964-af88-68b46ccd0897 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:20.423834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ecb2e-6564-4964-af88-68b46ccd0897 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:20.423834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ecb2e-6564-4964-af88-68b46ccd0897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:20.450143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fe9ecb2e-6564-4964-af88-68b46ccd0897 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:20.452055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fe9ecb2e-6564-4964-af88-68b46ccd0897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:20.461433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:20.618909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19143c-ffb6-439a-a7fc-79953ef01582 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:20.639182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19143c-ffb6-439a-a7fc-79953ef01582 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:20.646042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29740e-aa98-4f8f-a5c9-4d28a393451b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:20.646952 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29740e-aa98-4f8f-a5c9-4d28a393451b service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:20.647331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29740e-aa98-4f8f-a5c9-4d28a393451b service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:20.647806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29740e-aa98-4f8f-a5c9-4d28a393451b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:20.661993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29740e-aa98-4f8f-a5c9-4d28a393451b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:20.663805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29740e-aa98-4f8f-a5c9-4d28a393451b service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:20.674867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 772/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:21.502950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926700d-ee5a-486e-a593-4bc6bc42b701 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:21.503564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926700d-ee5a-486e-a593-4bc6bc42b701 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:21.503564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926700d-ee5a-486e-a593-4bc6bc42b701 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:21.504897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 15:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:21.511019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571205ee-3616-4203-91ee-84e33f997a22 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:21.511735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571205ee-3616-4203-91ee-84e33f997a22 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:21.511873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571205ee-3616-4203-91ee-84e33f997a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:21.540059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-571205ee-3616-4203-91ee-84e33f997a22 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:21.541274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-571205ee-3616-4203-91ee-84e33f997a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:21.552617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:21.714198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926700d-ee5a-486e-a593-4bc6bc42b701 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:21.736320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926700d-ee5a-486e-a593-4bc6bc42b701 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:21.742774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2787e5-9ecc-4b9b-9fe0-a1010025fbcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:21.743428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2787e5-9ecc-4b9b-9fe0-a1010025fbcb service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:21.743741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2787e5-9ecc-4b9b-9fe0-a1010025fbcb service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:21.744016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2787e5-9ecc-4b9b-9fe0-a1010025fbcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:21.752838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2787e5-9ecc-4b9b-9fe0-a1010025fbcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:21.754446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2787e5-9ecc-4b9b-9fe0-a1010025fbcb service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:21.764622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 754/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:25.889356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179eb7-becc-4682-8a0d-56dc05b2523c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:25.889951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179eb7-becc-4682-8a0d-56dc05b2523c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:25.890194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179eb7-becc-4682-8a0d-56dc05b2523c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:25.892657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:24:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:25.898261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c56a97-6e25-4891-b87c-4683e773b3bd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:25.898753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c56a97-6e25-4891-b87c-4683e773b3bd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:25.898869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c56a97-6e25-4891-b87c-4683e773b3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:25.918916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-66c56a97-6e25-4891-b87c-4683e773b3bd None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:25.920427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-66c56a97-6e25-4891-b87c-4683e773b3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:25.931738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:25.972738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf179eb7-becc-4682-8a0d-56dc05b2523c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:25.974746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf179eb7-becc-4682-8a0d-56dc05b2523c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:26.072655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf179eb7-becc-4682-8a0d-56dc05b2523c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:26.076292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d566d-7316-4c13-819a-cb4422110b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:26.076945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d566d-7316-4c13-819a-cb4422110b5b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:26.077226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d566d-7316-4c13-819a-cb4422110b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:26.077502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d566d-7316-4c13-819a-cb4422110b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:26.088111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d566d-7316-4c13-819a-cb4422110b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:26.089583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d566d-7316-4c13-819a-cb4422110b5b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:26.100251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 775/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:28.222736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1284416-fdf5-4e11-9def-bf48fb35dccf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:28.223383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1284416-fdf5-4e11-9def-bf48fb35dccf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:28.223383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1284416-fdf5-4e11-9def-bf48fb35dccf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:28.224669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:28.230624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b6ad-3fda-432c-b2a7-e504679dd7da None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:28.231078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b6ad-3fda-432c-b2a7-e504679dd7da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:28.231196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72b6ad-3fda-432c-b2a7-e504679dd7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:28.252134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6a72b6ad-3fda-432c-b2a7-e504679dd7da None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:28.253386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a72b6ad-3fda-432c-b2a7-e504679dd7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:28.262778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:28.300066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1284416-fdf5-4e11-9def-bf48fb35dccf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:28.318578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1284416-fdf5-4e11-9def-bf48fb35dccf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:28.325479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072d261-7fed-4e0f-83f1-920e39caaeaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:28.326318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072d261-7fed-4e0f-83f1-920e39caaeaa service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:28.326832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072d261-7fed-4e0f-83f1-920e39caaeaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:28.327316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072d261-7fed-4e0f-83f1-920e39caaeaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:28.346683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072d261-7fed-4e0f-83f1-920e39caaeaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:28.349383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072d261-7fed-4e0f-83f1-920e39caaeaa service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:28.368823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 757/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:32.059065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4550f5-3cd7-4e2e-99d3-8e2caf2e104c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:32.060328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4550f5-3cd7-4e2e-99d3-8e2caf2e104c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:32.060328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4550f5-3cd7-4e2e-99d3-8e2caf2e104c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:32.061253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:32.068998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffea218-0d20-453b-bba5-d516192b203d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:32.069737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffea218-0d20-453b-bba5-d516192b203d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:32.069737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffea218-0d20-453b-bba5-d516192b203d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:32.096076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dffea218-0d20-453b-bba5-d516192b203d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:32.097324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dffea218-0d20-453b-bba5-d516192b203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:32.105530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:32.426547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4550f5-3cd7-4e2e-99d3-8e2caf2e104c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:32.428637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5f4550f5-3cd7-4e2e-99d3-8e2caf2e104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:32.501492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:24:32.517367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4550f5-3cd7-4e2e-99d3-8e2caf2e104c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:32.521243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781be759-c8bb-4e46-b7f9-dbb706c7d40a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:32.521921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be759-c8bb-4e46-b7f9-dbb706c7d40a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:32.522310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be759-c8bb-4e46-b7f9-dbb706c7d40a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:32.522718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be759-c8bb-4e46-b7f9-dbb706c7d40a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:32.533486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781be759-c8bb-4e46-b7f9-dbb706c7d40a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:32.535066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781be759-c8bb-4e46-b7f9-dbb706c7d40a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:32.546578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 778/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:34.654009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02e323-c5bf-47f2-8835-4a58e606df21 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:34.654654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02e323-c5bf-47f2-8835-4a58e606df21 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:34.654654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02e323-c5bf-47f2-8835-4a58e606df21 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:34.656133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 15:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:24:34.661642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed36dc-59d8-4fd1-bfb6-5152414ba3a9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:34.662125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed36dc-59d8-4fd1-bfb6-5152414ba3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:34.662239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed36dc-59d8-4fd1-bfb6-5152414ba3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:34.682960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-55ed36dc-59d8-4fd1-bfb6-5152414ba3a9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:24:34.684315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-55ed36dc-59d8-4fd1-bfb6-5152414ba3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:34.694146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:24:35.070697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02e323-c5bf-47f2-8835-4a58e606df21 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:24:35.073539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2f02e323-c5bf-47f2-8835-4a58e606df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:24:35.094670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02e323-c5bf-47f2-8835-4a58e606df21 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:24:35.099393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9a501-9bee-4d8c-b12e-8082d0621fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:24:35.100063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9a501-9bee-4d8c-b12e-8082d0621fbd service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:24:35.100344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9a501-9bee-4d8c-b12e-8082d0621fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:24:35.100615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9a501-9bee-4d8c-b12e-8082d0621fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:24:35.111169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9a501-9bee-4d8c-b12e-8082d0621fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:24:35.112646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9a501-9bee-4d8c-b12e-8082d0621fbd service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:24:35.123614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 760/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:24:44.677694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=11 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:24:45.118610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:06.649641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:06.650950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:06.651462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:06.682942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:06.684409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:06.699910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 780/1540] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:06.708596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:06.709470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:06.709470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:06.739442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:06.740700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:06.754559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 761/1541] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:06.759821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:06.777508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dd542-4358-4968-8a35-6ae7c2c0ce5c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:06.788378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:06.789177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:06.789528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:06.789803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:06.790587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:06.790903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:06.792572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:06.817039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 781/1542] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:06.827698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:06.843448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d4e07-ba5d-4d86-98d7-631e8d687787 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:06.849109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58bb87a-2f46-4509-b841-9a99e925869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:06.849797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bb87a-2f46-4509-b841-9a99e925869b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:06.850078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bb87a-2f46-4509-b841-9a99e925869b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:06.850347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bb87a-2f46-4509-b841-9a99e925869b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:06.851237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bb87a-2f46-4509-b841-9a99e925869b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:06.853678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58bb87a-2f46-4509-b841-9a99e925869b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:06.918124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 762/1543] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:06 2022] POST /identity/v3/projects => generated 401 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:06.922140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:06.949347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58186589-e524-43b7-b3c5-1f94a376993c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.002604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ed46b-666a-450f-b0e6-3114a4810d0d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.002938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ed46b-666a-450f-b0e6-3114a4810d0d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.003798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ed46b-666a-450f-b0e6-3114a4810d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.059751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8f0ed46b-666a-450f-b0e6-3114a4810d0d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.061047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8f0ed46b-666a-450f-b0e6-3114a4810d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.114012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc730d-6656-4f56-b72b-453eab5db36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.114784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc730d-6656-4f56-b72b-453eab5db36d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.115074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc730d-6656-4f56-b72b-453eab5db36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.115362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc730d-6656-4f56-b72b-453eab5db36d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.116128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc730d-6656-4f56-b72b-453eab5db36d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.118436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc730d-6656-4f56-b72b-453eab5db36d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.163743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 782/1544] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:06 2022] POST /identity/v3/users => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:26:07.164989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.165356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.165526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.170161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 763/1545] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:26:07.171446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204090ad-ff26-41e2-b6f4-ff9a4a1f62a0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.171771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204090ad-ff26-41e2-b6f4-ff9a4a1f62a0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.172062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204090ad-ff26-41e2-b6f4-ff9a4a1f62a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.214397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.215779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.224511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-204090ad-ff26-41e2-b6f4-ff9a4a1f62a0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.225915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-204090ad-ff26-41e2-b6f4-ff9a4a1f62a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.246752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 783/1546] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.250253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 764/1547] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.251693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729781c-44c1-4ae6-9c82-d84e553477ce None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.252089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729781c-44c1-4ae6-9c82-d84e553477ce None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.252268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729781c-44c1-4ae6-9c82-d84e553477ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.253227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.277152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b540-6ee7-4593-9f17-4f84be2aa8f3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.293545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e5b0-e13c-4820-9795-4eff46438c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.295592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e5b0-e13c-4820-9795-4eff46438c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.295946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e5b0-e13c-4820-9795-4eff46438c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.296233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e5b0-e13c-4820-9795-4eff46438c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.296938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e5b0-e13c-4820-9795-4eff46438c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.298702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e5b0-e13c-4820-9795-4eff46438c29 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.311015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8729781c-44c1-4ae6-9c82-d84e553477ce None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.313755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8729781c-44c1-4ae6-9c82-d84e553477ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.333080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 784/1548] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:07.334453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.334785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.335054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.343590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 765/1549] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.345935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.346254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.346545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.401481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.402778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.409020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.410245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.425355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 785/1550] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.426178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.432991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 766/1551] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.433900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.447084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1709798-30b3-440d-ab79-0898835a9e44 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.452899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4491be67-d607-4f9e-8771-d07564fc2652 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.459111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.459854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.460154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.460461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.461599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.463339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.469042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.469768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.470016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.470286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.471086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:07.471399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.472440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:26:07.472440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:26:07.472440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:26:07.473365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.494241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 786/1552] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:07.496290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.502356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 767/1553] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:07.503286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.518019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c8033-2a2b-4708-8f1b-12f5277d5838 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.523127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55937620-f9da-4799-8be6-b6fb866ccbe9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.524766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.525470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.526914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.527252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.527980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:07.528325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.530407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea52d54-41a5-4445-b205-48e4f1353e9e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.536793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.540503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.541619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.542940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.543746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:07.544066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.545811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.550886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 787/1554] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:07.554118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d54017-70f1-47d9-bdfc-e1640bf0168c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.555478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d54017-70f1-47d9-bdfc-e1640bf0168c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.556350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d54017-70f1-47d9-bdfc-e1640bf0168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.575204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 768/1555] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:07.578935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.582243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-80d54017-70f1-47d9-bdfc-e1640bf0168c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.583883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80d54017-70f1-47d9-bdfc-e1640bf0168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.594005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 788/1556] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.594991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.595207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.595513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.598877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf463c-9d48-4f5b-b446-74b3ef828c1f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.606350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.607295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.607574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.607848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] PATH_INFO: `/v3/projects/d0c12f9c72574a6db10c4d86676e0d03/users/ea717e5252ec4d6d8b3e293d9981f2b3/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.615377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.616637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.625333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 789/1557] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.626171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.635679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c12f9c72574a6db10c4d86676e0d03, user_id=ea717e5252ec4d6d8b3e293d9981f2b3, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.637043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f499cb1a-62ac-4506-954c-73428a4406e9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.639317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.644832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.645586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.645869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.646143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.646928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.648172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:26:07.648172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:26:07.648172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:26:07.649545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e242e7-203d-4942-8fcb-6b147cc35063 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.658006 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 769/1558] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:07 2022] PUT /identity/v3/projects/d0c12f9c72574a6db10c4d86676e0d03/users/ea717e5252ec4d6d8b3e293d9981f2b3/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:07.658894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.684099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 790/1559] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:07.686355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.686690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.686998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.706541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.707801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.723056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:07.735759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc154d-c133-49c1-88e2-3bac988d5000 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.759625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:07.787673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 791/1560] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.789844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.802191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c339d6-794f-4b88-abd7-5a8efc113427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.802478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14d9ea-6bb6-4fba-a5c3-e56d19d3fa61 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.802966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c339d6-794f-4b88-abd7-5a8efc113427 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.803273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c339d6-794f-4b88-abd7-5a8efc113427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.803558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c339d6-794f-4b88-abd7-5a8efc113427 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.804484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c339d6-794f-4b88-abd7-5a8efc113427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.806476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c339d6-794f-4b88-abd7-5a8efc113427 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.808561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0975e4-f14b-40ab-8adc-0db6ba15391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.809454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0975e4-f14b-40ab-8adc-0db6ba15391e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.809853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0975e4-f14b-40ab-8adc-0db6ba15391e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.810457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0975e4-f14b-40ab-8adc-0db6ba15391e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.811392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0975e4-f14b-40ab-8adc-0db6ba15391e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.813521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0975e4-f14b-40ab-8adc-0db6ba15391e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:07.844189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 770/1561] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/projects => generated 377 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:07.844189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5157-128f-4c03-b51e-92fb91523b2a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.844189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5157-128f-4c03-b51e-92fb91523b2a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.844189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e5157-128f-4c03-b51e-92fb91523b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.853580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 792/1562] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:07.853580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2676a-aef8-426f-b10f-d0a078154bb7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.853580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2676a-aef8-426f-b10f-d0a078154bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.853580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd2676a-aef8-426f-b10f-d0a078154bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.864141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-009e5157-128f-4c03-b51e-92fb91523b2a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.865386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-009e5157-128f-4c03-b51e-92fb91523b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.875276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-acd2676a-aef8-426f-b10f-d0a078154bb7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.876522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-acd2676a-aef8-426f-b10f-d0a078154bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.928903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 771/1563] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.930296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.930721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.930900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.952661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 793/1564] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.954168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e10063-bba0-4f94-b6b6-ce5a8dfa794c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.954287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e10063-bba0-4f94-b6b6-ce5a8dfa794c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.954599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e10063-bba0-4f94-b6b6-ce5a8dfa794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.958474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.960374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.970989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 772/1565] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.971820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:07.974505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-55e10063-bba0-4f94-b6b6-ce5a8dfa794c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:07.975750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-55e10063-bba0-4f94-b6b6-ce5a8dfa794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:07.982372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8377d-c09d-49fb-9b1f-9e4d58800bf2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:07.983790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 794/1566] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:07.985013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.985287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.985560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.987207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:07.988012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:07.988180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:07.988434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:07.989169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:07.990201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:26:07.990201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:26:07.990201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:26:07.991459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.008835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:08.010121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:08.018265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 773/1567] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.018265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.019228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 795/1568] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:08.020058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.030109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357690c-569a-4377-b486-bb3cc81d24e3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.030740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c42b1e-5b21-44a1-a9a4-531744055238 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.034993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60aed35-51be-4bab-a48d-4e00e922d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.035683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aed35-51be-4bab-a48d-4e00e922d96f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.035970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aed35-51be-4bab-a48d-4e00e922d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.036171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.036394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aed35-51be-4bab-a48d-4e00e922d96f admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.036882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.037064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60aed35-51be-4bab-a48d-4e00e922d96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.037226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.037410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.038092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.039059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60aed35-51be-4bab-a48d-4e00e922d96f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.040274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.060496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 774/1569] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.062267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.062589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.063012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.063529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 796/1570] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/users => generated 294 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.067419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.078702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9ae95-e093-4f34-9841-521336ad30b2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.081676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:08.082949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:08.084570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.085582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.086137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.086683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.087545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:08.087873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.089594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.091627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 775/1571] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:08.092489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.102886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64438927-6827-48e5-8c3a-271b5dbc55bf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.107352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 797/1572] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:08.107542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.108170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.108296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.108664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.108776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.109679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:08.110019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.111908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.127474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 776/1573] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:08.128401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.139706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f744b-282c-4a8a-bc61-4753dae3d15e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.144601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.145287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.145611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.145986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.146802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.148543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.161489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 777/1574] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:08.162368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.173920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca205-9a70-4443-b86c-137db323d22e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.178865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055465-f204-4719-a740-658a332f964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.179578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055465-f204-4719-a740-658a332f964d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.180609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055465-f204-4719-a740-658a332f964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.180904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72055465-f204-4719-a740-658a332f964d admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.181674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055465-f204-4719-a740-658a332f964d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:08.182436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055465-f204-4719-a740-658a332f964d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.183699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72055465-f204-4719-a740-658a332f964d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.186799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989b167-daf7-4eed-aa9f-83a51a33e717 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.190868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.191532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.191811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.192084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.193835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:08.200312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 778/1575] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:08.203744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055465-f204-4719-a740-658a332f964d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.206421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.209261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.217153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72055465-f204-4719-a740-658a332f964d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.220537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 798/1576] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:08.221390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.222430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.223152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.223437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.223710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.224646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.227580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.232709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4557804-a40f-4846-a7c8-49dfc39531cc service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.237740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.238417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.238752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.242213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.242759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 779/1577] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:08.242978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.243168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.243275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.254753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d10885-f91b-4e8b-9add-18d742443857 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.255551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 799/1578] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:08.256180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.264274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.272924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.272924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.272924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.273868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87547b3e-4767-4855-99cd-3fba08b7b54f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.273979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:08.273979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.273979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.279526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.280209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.280492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.280768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.281537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.283627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.284645 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 780/1579] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:08.285619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.297318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0effbbf-26a5-40a1-818f-bf85b3161843 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.298801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:08.305472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.306284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.306603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.306888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.307646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.309843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.332905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 800/1580] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.334133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.347860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 781/1581] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/projects => generated 393 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.351719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.445621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb5db6-3192-4cab-a0a5-6798a5335e86 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.448721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8874c71-5599-4079-b60c-4228b10e519c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.454159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.455313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.455895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.456576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.456865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.457751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.458132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.458302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.458711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.459125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.461388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.462395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.475867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 801/1582] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:08.477016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.489346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83ee0f-ffcf-4f9b-8b4a-d1305f7b2c2e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.496239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.498286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.498629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.498913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.502987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 782/1583] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.503165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.503271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.503503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.532293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 802/1584] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:08.532293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.550843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b59b0b-bd0d-40b7-b7cc-a02a7817e81d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.615578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638b3d5-8c50-42cc-a7cd-60eb7757b711 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.621709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.622406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.622859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.623130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.623910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.625588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.642088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 783/1585] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:08.642088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.658017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a137b3f-e47f-478f-b27b-544f10f8f5fb admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.664902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.665805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.666168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.666549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.667514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.669636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.672829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.673838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.674368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.674873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.675987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.679329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.704012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 784/1586] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/users => generated 287 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.704012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.718215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a791415-ee30-41ae-aed0-e680ddffff9f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.724126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 803/1587] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/projects => generated 379 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.724704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.725158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.725251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.725331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.725418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.726077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.728649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.757102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:08.768203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 785/1588] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/users => generated 297 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.770232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.787463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415faf3a-c005-44eb-a98f-6af6fda26eb2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.844918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417049-6c4a-40d7-ab04-500f4044cd26 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.850499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.851628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.851988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.852282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] PATH_INFO: `/v3/projects/1b2864575fba44f29d22c7afb207a73b/users/5e6af511fedd4788b1b771db772251f3/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.856339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:08.865357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.866358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.867106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.867106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:08.867903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.870103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.874800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2864575fba44f29d22c7afb207a73b, user_id=5e6af511fedd4788b1b771db772251f3, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:08.876144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:26:08.876144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:26:08.876144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:26:08.879928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:08.893673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 786/1589] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:08 2022] POST /identity/v3/users => generated 282 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:08.894760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.902202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 804/1590] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:08 2022] PUT /identity/v3/projects/1b2864575fba44f29d22c7afb207a73b/users/5e6af511fedd4788b1b771db772251f3/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:08.903525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:08.910440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2daefb-616d-4a85-90be-36039496b4da admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:08.988814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:08.989719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:08.989998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:08.990276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] PATH_INFO: `/v3/projects/5e51044e804442909f18288b9a6430a5/users/08050c1635084492a4b054b172884af8/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.010584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b577821-7a63-44a2-8136-71cf17c3c9d9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.011725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e51044e804442909f18288b9a6430a5, user_id=08050c1635084492a4b054b172884af8, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.015038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.015604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.016508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.016785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.017061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] PATH_INFO: `/v3/projects/83cbbde066834f25a1b6f1bdc28e8b79/users/9a0d49b3b50b4e668e157cb2e8e47da2/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.036943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 787/1591] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:08 2022] PUT /identity/v3/projects/5e51044e804442909f18288b9a6430a5/users/08050c1635084492a4b054b172884af8/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:09.038180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.044341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83cbbde066834f25a1b6f1bdc28e8b79, user_id=9a0d49b3b50b4e668e157cb2e8e47da2, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.047325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.063059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 805/1592] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:08 2022] PUT /identity/v3/projects/83cbbde066834f25a1b6f1bdc28e8b79/users/9a0d49b3b50b4e668e157cb2e8e47da2/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:09.067506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.138853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211b72f-56b7-4dc2-9b88-0d6de2d1e05b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.164312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:09.178226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1878cc-781b-44ca-9deb-8e1168e81f61 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.188391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.189513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.189513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.189833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.190563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.192363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.221508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.222103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.222380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.222850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.225566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.225763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 806/1593] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:09.225763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.229506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.236534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96967f3-f760-48c6-9e8a-51bc72863315 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.240707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 788/1594] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.242017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.244876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf2997-830a-4b5a-a74a-6a117779b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.246295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2997-830a-4b5a-a74a-6a117779b9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.247009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2997-830a-4b5a-a74a-6a117779b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.247525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf2997-830a-4b5a-a74a-6a117779b9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.248613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf2997-830a-4b5a-a74a-6a117779b9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.250563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccf2997-830a-4b5a-a74a-6a117779b9d4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.255053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2991c-3fb3-4777-bfc4-0ae41a957355 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.259967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f87dd-e091-4499-b66e-e5ed79cb4299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.260748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f87dd-e091-4499-b66e-e5ed79cb4299 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.261122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f87dd-e091-4499-b66e-e5ed79cb4299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.261585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f87dd-e091-4499-b66e-e5ed79cb4299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.265070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 807/1595] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.265225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f87dd-e091-4499-b66e-e5ed79cb4299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.265334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ffbb9-53d9-4cd5-9fe6-4fadd158fd9c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.265334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ffbb9-53d9-4cd5-9fe6-4fadd158fd9c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.265334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ffbb9-53d9-4cd5-9fe6-4fadd158fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.265608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f87dd-e091-4499-b66e-e5ed79cb4299 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.275958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 789/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.280352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e293b-9c23-4802-9182-913f4cc29fdf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.280822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e293b-9c23-4802-9182-913f4cc29fdf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.281106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04e293b-9c23-4802-9182-913f4cc29fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.292927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1f1ffbb9-53d9-4cd5-9fe6-4fadd158fd9c None None] MFA Rules not processed for user `5e6af511fedd4788b1b771db772251f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:09.294245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1f1ffbb9-53d9-4cd5-9fe6-4fadd158fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.301028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a04e293b-9c23-4802-9182-913f4cc29fdf None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:09.302287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a04e293b-9c23-4802-9182-913f4cc29fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.310574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 790/1597] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:09.312201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.312201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.312601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.331009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:09.345599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] MFA Rules not processed for user `ea717e5252ec4d6d8b3e293d9981f2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:09.347666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.369140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 808/1598] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:09.370822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.370936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.371186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.391364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:09.392762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.402629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 809/1599] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:09.403401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.421234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdb1df-befd-4463-94d2-24cb9861a194 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.424392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 791/1600] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:09.425400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.430552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.431409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.431691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.432481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.432732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.434549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.436157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c6aab-7ade-47ce-8fa2-a2c84902126b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.440700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.441369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.441650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.441923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.442792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.444407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.450438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 810/1601] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.450438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.456807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 792/1602] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.457722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.458561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b8a88-63b5-4b91-aaa4-1428691ca0a8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.467260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.468110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.468507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.468945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.470030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.471476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93618184-0977-4994-81d8-ca95a146b9fe admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.472064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.478541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c0b46e-fa8d-4de8-b91a-cd8b385b17f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.479693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0b46e-fa8d-4de8-b91a-cd8b385b17f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.480104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0b46e-fa8d-4de8-b91a-cd8b385b17f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.480279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c0b46e-fa8d-4de8-b91a-cd8b385b17f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.481017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0b46e-fa8d-4de8-b91a-cd8b385b17f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.482686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c0b46e-fa8d-4de8-b91a-cd8b385b17f2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.484157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 811/1603] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.485006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.486928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.501060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 793/1604] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.502550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a635e4-54d8-443c-9c53-79341a054f85 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.502877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a635e4-54d8-443c-9c53-79341a054f85 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.503160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a635e4-54d8-443c-9c53-79341a054f85 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.504556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 794/1605] 10.222.0.40 () {54 vars in 961 bytes} [Fri Jul 15 15:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:09.505316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a635e4-54d8-443c-9c53-79341a054f85 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.509729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c5a635e4-54d8-443c-9c53-79341a054f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.552041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7686ea19-c88c-4425-8d4c-1dac6d038ea0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.557077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.557794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.558075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.558475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.573220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.575556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.578894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a635e4-54d8-443c-9c53-79341a054f85 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.582650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.583312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.583598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.583884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.590249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 812/1606] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:09.591468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.594247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.595761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.603959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95303eb1-ace4-4421-b1a8-e4a0032f78a9 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.607923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 795/1607] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:09.608966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.610578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.611589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.612086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.612665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.613597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.615796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.621955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bfa5e-57bf-4aa6-907c-3f03d468b8d7 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.634727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.635445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.635722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.636024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.636783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.638545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.642450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:09.655378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 796/1608] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:09.656423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.664965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 813/1609] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/projects => generated 405 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:09.664965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.671275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15373cb2-6ef4-40a2-81cc-5a4689298a70 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.759578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.761249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.761752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.762192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] PATH_INFO: `/v3/projects/154d83d3acca49b0b07aceb57c6fa4ed/users/1f963e5fd1474a9fb209410f22e783e8/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.787905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62617eb-a1bd-4356-ae8b-68fdaaa5f8ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.791117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=154d83d3acca49b0b07aceb57c6fa4ed, user_id=1f963e5fd1474a9fb209410f22e783e8, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.795674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.797188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42214040-4ae6-4365-b5d7-b67ca633578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.798823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42214040-4ae6-4365-b5d7-b67ca633578e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.799288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42214040-4ae6-4365-b5d7-b67ca633578e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.799947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42214040-4ae6-4365-b5d7-b67ca633578e admin admin] PATH_INFO: `/v3/projects/1e21747a454c4ebebb5d48b5906d5e5c/users/c1a1af21d3f94b70a830b14200619f87/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.819956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 797/1610] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:09 2022] PUT /identity/v3/projects/154d83d3acca49b0b07aceb57c6fa4ed/users/1f963e5fd1474a9fb209410f22e783e8/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:09.821422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:09.829225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42214040-4ae6-4365-b5d7-b67ca633578e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e21747a454c4ebebb5d48b5906d5e5c, user_id=c1a1af21d3f94b70a830b14200619f87, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:09.834957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42214040-4ae6-4365-b5d7-b67ca633578e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:09.859934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 814/1611] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:09 2022] PUT /identity/v3/projects/1e21747a454c4ebebb5d48b5906d5e5c/users/c1a1af21d3f94b70a830b14200619f87/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:09.859934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.859934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.859934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:09.882030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:09.906901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50a654-f38f-47ad-936f-9a8742cb7cb3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:09.909717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:09.911104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:09.987336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:09.988772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:09.988772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:09.989190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] PATH_INFO: `/v3/projects/c1ed18ddda3846ad971faedc3cd6dfb7/users/baad6e8d21fe4d40ab2fc0960438efb0/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.008211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ed18ddda3846ad971faedc3cd6dfb7, user_id=baad6e8d21fe4d40ab2fc0960438efb0, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.011394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.030786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 798/1612] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:09 2022] PUT /identity/v3/projects/c1ed18ddda3846ad971faedc3cd6dfb7/users/baad6e8d21fe4d40ab2fc0960438efb0/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:10.030786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.033333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 815/1613] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Jul 15 15:26:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:10.034367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.053242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15727932-11a0-40fb-9579-82ee67acbc1f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.061273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:10.122881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30d7bc-deae-420c-b50d-5224ac332ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:10.124012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d7bc-deae-420c-b50d-5224ac332ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.124505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d7bc-deae-420c-b50d-5224ac332ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.125160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30d7bc-deae-420c-b50d-5224ac332ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.125675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2ded-0444-45b7-bc45-84f663036c88 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.126361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d7bc-deae-420c-b50d-5224ac332ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.128757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30d7bc-deae-420c-b50d-5224ac332ab2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.131707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34efe2d-3761-46f3-abe4-6faa13503fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:10.132666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34efe2d-3761-46f3-abe4-6faa13503fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.132943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34efe2d-3761-46f3-abe4-6faa13503fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.133249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34efe2d-3761-46f3-abe4-6faa13503fc0 admin admin] PATH_INFO: `/v3/projects/f1097bdacbb44067bfa5dbf28bc86d9a/users/2a01407bfc0a480a8f6702993ddb0183/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.150549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34efe2d-3761-46f3-abe4-6faa13503fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1097bdacbb44067bfa5dbf28bc86d9a, user_id=2a01407bfc0a480a8f6702993ddb0183, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.153523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34efe2d-3761-46f3-abe4-6faa13503fc0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.154998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 816/1614] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/users => generated 309 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:10.156780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.157076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.157348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.177287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 799/1615] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:10 2022] PUT /identity/v3/projects/f1097bdacbb44067bfa5dbf28bc86d9a/users/2a01407bfc0a480a8f6702993ddb0183/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:10.179052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e3544-7c6f-452e-899b-f8510680b3c0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.179452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e3544-7c6f-452e-899b-f8510680b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.179800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e3544-7c6f-452e-899b-f8510680b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.193833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] MFA Rules not processed for user `1f963e5fd1474a9fb209410f22e783e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:10.196207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:10.228928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4a6e3544-7c6f-452e-899b-f8510680b3c0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:10.230177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4a6e3544-7c6f-452e-899b-f8510680b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:10.278579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 817/1616] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:10.279907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.292079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:10.330314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 800/1617] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:10.332373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.334194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.336018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.383678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] MFA Rules not processed for user `baad6e8d21fe4d40ab2fc0960438efb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:10.385026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:10.442258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfefe2-4e3d-4eb9-a5d3-2d80d17be25f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.449346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97741441-d454-4ca1-a6c0-1896383d0c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:10.449993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741441-d454-4ca1-a6c0-1896383d0c93 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.450278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741441-d454-4ca1-a6c0-1896383d0c93 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.450581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97741441-d454-4ca1-a6c0-1896383d0c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.540646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 801/1618] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:10.542240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.560338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97741441-d454-4ca1-a6c0-1896383d0c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.562196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97741441-d454-4ca1-a6c0-1896383d0c93 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.568229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad08294-fc9d-471e-a05e-d438d0bb95bd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.574702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84733f-4527-4328-9c84-8f9714108eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:10.575766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84733f-4527-4328-9c84-8f9714108eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.576280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84733f-4527-4328-9c84-8f9714108eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.576832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84733f-4527-4328-9c84-8f9714108eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.577839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84733f-4527-4328-9c84-8f9714108eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.579794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84733f-4527-4328-9c84-8f9714108eb4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.597516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 818/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:10.600563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.600824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.601229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.601896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 802/1620] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:10.604114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90158d-bd2e-4e20-8124-ea43c47de6d5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.604662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90158d-bd2e-4e20-8124-ea43c47de6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.605161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90158d-bd2e-4e20-8124-ea43c47de6d5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.607087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 803/1621] 10.222.0.40 () {54 vars in 961 bytes} [Fri Jul 15 15:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:10.615983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90158d-bd2e-4e20-8124-ea43c47de6d5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.638236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:10.656218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:10.657618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:10.684683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 819/1622] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:10.689152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.702830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee22c5-5a53-4cb6-bbce-bb65adc3d026 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.705641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90158d-bd2e-4e20-8124-ea43c47de6d5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.711024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c9f48-a415-41d5-88d7-11524bef6343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:10.711298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:10.712393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c9f48-a415-41d5-88d7-11524bef6343 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.713394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.713830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.714349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c9f48-a415-41d5-88d7-11524bef6343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.715157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.715610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c9f48-a415-41d5-88d7-11524bef6343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.716861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c9f48-a415-41d5-88d7-11524bef6343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.720714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c9f48-a415-41d5-88d7-11524bef6343 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.741018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:10.745971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:10.782484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 804/1623] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:10.785187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:10.800710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 820/1624] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:10.807402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:10.807713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:10.809285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:10.815547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9c19f-2a5b-4d4c-8dc0-4ce0280ae91d service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:10.854711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:10.857191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:11.011235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.012195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.012601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.012894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] PATH_INFO: `/v3/projects/63772087b05f40a6b22d6f4b8f3d07c5/users/717eb4a29d41488a8a0937fdcf44d0db/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.033988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63772087b05f40a6b22d6f4b8f3d07c5, user_id=717eb4a29d41488a8a0937fdcf44d0db, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.036830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.060645 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 805/1625] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:10 2022] PUT /identity/v3/projects/63772087b05f40a6b22d6f4b8f3d07c5/users/717eb4a29d41488a8a0937fdcf44d0db/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:11.060645 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.062504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 821/1626] 10.222.0.40 () {58 vars in 1058 bytes} [Fri Jul 15 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:11.062955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.081671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068e1545-5e48-4330-86d4-0fea6070e94e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:11.088629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:11.206411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9b4a1-83df-44d0-8f3c-f2c8b27b2f4c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:11.213555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a548e-9616-4720-a9ed-70c2435ecf87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.214438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a548e-9616-4720-a9ed-70c2435ecf87 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.215029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a548e-9616-4720-a9ed-70c2435ecf87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.215414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a548e-9616-4720-a9ed-70c2435ecf87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.223117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.223815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.224099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.224378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.225087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.227840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.232951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a548e-9616-4720-a9ed-70c2435ecf87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.235647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a548e-9616-4720-a9ed-70c2435ecf87 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.257324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 806/1627] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:11.260472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fc90d-83ad-4b83-9db8-f6d426f1201d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.261105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fc90d-83ad-4b83-9db8-f6d426f1201d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.262044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fc90d-83ad-4b83-9db8-f6d426f1201d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.264564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 807/1628] 10.222.0.40 () {54 vars in 912 bytes} [Fri Jul 15 15:26:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:11.266181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fc90d-83ad-4b83-9db8-f6d426f1201d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.269938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 822/1629] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:11 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:11.275515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.331433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fc90d-83ad-4b83-9db8-f6d426f1201d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:11.368197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcd37f-23d0-4406-a97b-997cd9508c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.369732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd37f-23d0-4406-a97b-997cd9508c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.371353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd37f-23d0-4406-a97b-997cd9508c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.372102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcd37f-23d0-4406-a97b-997cd9508c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.373766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd37f-23d0-4406-a97b-997cd9508c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.378497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcd37f-23d0-4406-a97b-997cd9508c83 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.427439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 808/1630] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:11.432655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.433508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.434187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.462775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac110da-00ca-4379-b2e8-df60823400f1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:11.499180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.507069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.508984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.510399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.532239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:11.534805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:11.553671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 809/1631] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:26:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:11.556362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.583005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824a523-b4dd-47de-870f-2d3fbfe06cff None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:11.593509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.595155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.595816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.596834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.598669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.603011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.639413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 810/1632] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:11.642106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.671889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.674655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.679729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34074d6e-e5ef-4ed4-bd16-40bdd7c8786b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:11.701972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:11.706282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:11.707900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:11.709494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] PATH_INFO: `/v3/projects/63772087b05f40a6b22d6f4b8f3d07c5/users/717eb4a29d41488a8a0937fdcf44d0db/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:11.720750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 823/1633] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:11.721412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.723939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63772087b05f40a6b22d6f4b8f3d07c5, user_id=717eb4a29d41488a8a0937fdcf44d0db, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:11.737638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:11.764440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:11.794326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 811/1634] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:11 2022] PUT /identity/v3/projects/63772087b05f40a6b22d6f4b8f3d07c5/users/717eb4a29d41488a8a0937fdcf44d0db/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:11.797712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:11.801802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:11.882911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a0834-ecaa-4645-8bc9-f4ac73013ad9 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:12.084531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c03db-bba5-4751-b523-b810a40dd499 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:12.099111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:12.100306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.100781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.101856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] PATH_INFO: `/v3/projects/3e116f54062843169dbffbfa130584a0/users/eeb56e922e33492bb8d04e68b607a7cd/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.112590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:12.113284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.115865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.116146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.163755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e116f54062843169dbffbfa130584a0, user_id=eeb56e922e33492bb8d04e68b607a7cd, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:12.172125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:12.208691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 812/1635] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:11 2022] PUT /identity/v3/projects/3e116f54062843169dbffbfa130584a0/users/eeb56e922e33492bb8d04e68b607a7cd/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:12.215176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:12.269053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:12.271055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:12.291165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 824/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:12.353310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874730a-ac97-4b3e-9632-e093a5d37180 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:12.363928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f3040-4290-4c3c-b577-beeda3c13fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:12.375721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3040-4290-4c3c-b577-beeda3c13fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.376568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3040-4290-4c3c-b577-beeda3c13fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.377040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f3040-4290-4c3c-b577-beeda3c13fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.379299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f3040-4290-4c3c-b577-beeda3c13fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:12.383877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f3040-4290-4c3c-b577-beeda3c13fe0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:12.416618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 813/1637] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:12.422877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:12.442076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1db5f5-c1d1-40d0-ab10-a705c2a32eb7 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:12.449869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902195a1-4a84-46b1-80e0-6d4867a124d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:12.451094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902195a1-4a84-46b1-80e0-6d4867a124d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.451559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902195a1-4a84-46b1-80e0-6d4867a124d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.451848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902195a1-4a84-46b1-80e0-6d4867a124d7 admin admin] PATH_INFO: `/v3/projects/3e116f54062843169dbffbfa130584a0/users/eeb56e922e33492bb8d04e68b607a7cd/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.454052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902195a1-4a84-46b1-80e0-6d4867a124d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e116f54062843169dbffbfa130584a0, user_id=eeb56e922e33492bb8d04e68b607a7cd, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:12.461345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902195a1-4a84-46b1-80e0-6d4867a124d7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:12.480463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 825/1638] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:12 2022] PUT /identity/v3/projects/3e116f54062843169dbffbfa130584a0/users/eeb56e922e33492bb8d04e68b607a7cd/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:12.501496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.502467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.503096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.550541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] MFA Rules not processed for user `eeb56e922e33492bb8d04e68b607a7cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:12.551960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:12.570304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc235b1-a55b-4ee6-8af3-914fbef80fef None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.570970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc235b1-a55b-4ee6-8af3-914fbef80fef None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.571147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc235b1-a55b-4ee6-8af3-914fbef80fef None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.573280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 826/1639] 10.222.0.40 () {54 vars in 912 bytes} [Fri Jul 15 15:26:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:12.591449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.591449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.592011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.637168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:12.640394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:12.770223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 814/1640] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:12.791155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:12.798203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:12.841249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 827/1641] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:26:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:26:12.931434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b6715-280a-4259-b94e-7d29a1d6467e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:12.946232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:12.947699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:12.948128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:12.948547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:12.964387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:12.966188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:12.997345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 815/1642] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:13.147503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:13.179985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe6640-183c-4c0d-af51-4443b3d3feb9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:13.199632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6feed-87e7-43bb-b15d-882ddc5c079c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:13.200352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6feed-87e7-43bb-b15d-882ddc5c079c service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:13.201693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6feed-87e7-43bb-b15d-882ddc5c079c service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:13.203864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6feed-87e7-43bb-b15d-882ddc5c079c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:13.238451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6feed-87e7-43bb-b15d-882ddc5c079c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:13.243218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6feed-87e7-43bb-b15d-882ddc5c079c service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:13.269139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:13.300879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 828/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:26:13.607267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:13.618169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7831f8-7164-4ef8-9b27-3598bf66e342 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:13.623680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d89b4f-84c4-4b14-84be-48ba4fb64f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:13.624800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d89b4f-84c4-4b14-84be-48ba4fb64f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:13.625218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d89b4f-84c4-4b14-84be-48ba4fb64f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:13.626127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d89b4f-84c4-4b14-84be-48ba4fb64f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:13.833473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d89b4f-84c4-4b14-84be-48ba4fb64f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:13.834511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d89b4f-84c4-4b14-84be-48ba4fb64f66 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:13.872950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 816/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:14.297124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8641db3-5d51-4511-84f3-82a044133fb5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:14.297716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8641db3-5d51-4511-84f3-82a044133fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:14.298557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8641db3-5d51-4511-84f3-82a044133fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:14.357303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e8641db3-5d51-4511-84f3-82a044133fb5 None None] MFA Rules not processed for user `717eb4a29d41488a8a0937fdcf44d0db`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:14.358744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8641db3-5d51-4511-84f3-82a044133fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:14.531329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 829/1645] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:18.304569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ac94f-fa25-4706-9af0-9b81daf8876d None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:18.305367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ac94f-fa25-4706-9af0-9b81daf8876d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:18.305367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ac94f-fa25-4706-9af0-9b81daf8876d None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:18.311038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 817/1646] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:26:18 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:18.343982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3d57-8077-4de0-b508-411b1b414cc6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:18.344742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3d57-8077-4de0-b508-411b1b414cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:18.346378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c3d57-8077-4de0-b508-411b1b414cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:18.399505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-335c3d57-8077-4de0-b508-411b1b414cc6 None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:18.401865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-335c3d57-8077-4de0-b508-411b1b414cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:18.570114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 830/1647] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:26:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:18.663896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3eda8-bfcc-4a66-a11b-1f621401f733 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:18.664480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3eda8-bfcc-4a66-a11b-1f621401f733 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:18.664568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3eda8-bfcc-4a66-a11b-1f621401f733 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:18.668994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 818/1648] 10.222.0.40 () {54 vars in 960 bytes} [Fri Jul 15 15:26:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:26:18.683867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b85ef-c79a-46f4-9136-cc7ffd60c79a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:18.684167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b85ef-c79a-46f4-9136-cc7ffd60c79a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:18.684433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b85ef-c79a-46f4-9136-cc7ffd60c79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:18.715653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9d0b85ef-c79a-46f4-9136-cc7ffd60c79a None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:18.716933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d0b85ef-c79a-46f4-9136-cc7ffd60c79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:18.739357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 831/1649] 10.222.0.40 () {58 vars in 1057 bytes} [Fri Jul 15 15:26:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:18.749354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3eda8-bfcc-4a66-a11b-1f621401f733 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:18.770454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3eda8-bfcc-4a66-a11b-1f621401f733 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:18.783368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc5508c-ac9b-4e0e-9df9-16869bebac38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:18.784074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5508c-ac9b-4e0e-9df9-16869bebac38 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:18.784359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5508c-ac9b-4e0e-9df9-16869bebac38 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:18.784634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc5508c-ac9b-4e0e-9df9-16869bebac38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:18.810354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5508c-ac9b-4e0e-9df9-16869bebac38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:18.812004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc5508c-ac9b-4e0e-9df9-16869bebac38 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:18.834574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 819/1650] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:20.724905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beca7d4-b929-4086-865a-4de1fa4d8372 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:20.725598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beca7d4-b929-4086-865a-4de1fa4d8372 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:20.725834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beca7d4-b929-4086-865a-4de1fa4d8372 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:20.728281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 832/1651] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:26:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:20.762406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04e975-84bd-484f-928a-d1bc1773f543 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:20.763481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04e975-84bd-484f-928a-d1bc1773f543 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:20.763830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04e975-84bd-484f-928a-d1bc1773f543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:20.812923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2e04e975-84bd-484f-928a-d1bc1773f543 None None] MFA Rules not processed for user `13549f6dc97241f28501b4c3e3c25959`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:20.814496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2e04e975-84bd-484f-928a-d1bc1773f543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:20.828247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 820/1652] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:20.978655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beca7d4-b929-4086-865a-4de1fa4d8372 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:21.003455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beca7d4-b929-4086-865a-4de1fa4d8372 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:21.022557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:21.023301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:21.023585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:21.023858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:21.045328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:21.047192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:21.077576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 833/1653] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:21.083479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:21.083767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:21.084043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:21.132197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] MFA Rules not processed for user `9a0d49b3b50b4e668e157cb2e8e47da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:21.133290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:21.138173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:21.266845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610baf3d-494b-4f42-b461-72c5add66c58 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:21.268009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 821/1654] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:21.272315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:21.273351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:21.273623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:21.273892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] PATH_INFO: `/v3/users/ea717e5252ec4d6d8b3e293d9981f2b3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:21.275191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea717e5252ec4d6d8b3e293d9981f2b3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:21.276256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:26:21.276256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: warnings.warn(msg) Jul 15 15:26:21.276256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: Jul 15 15:26:21.277851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:21.282581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:21.292391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:21.322251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9c10e-2d96-4cdc-9ff1-5d2b90476c6e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:21.330693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:21.331385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:21.331675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:21.331959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:21.347055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:21.348710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:21.364422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 822/1655] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:21.427389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea717e5252ec4d6d8b3e293d9981f2b3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:21.450856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea717e5252ec4d6d8b3e293d9981f2b3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:21.472440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea717e5252ec4d6d8b3e293d9981f2b3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:21.477359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea717e5252ec4d6d8b3e293d9981f2b3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:21.478771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea717e5252ec4d6d8b3e293d9981f2b3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:21.545046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 834/1656] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:21 2022] DELETE /identity/v3/users/ea717e5252ec4d6d8b3e293d9981f2b3 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:22.032835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:22.040765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:22.062200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c93d51-0bbc-4bbb-98d6-9608d80e8609 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:22.166218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:22.169882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:22.177697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:22.178160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] PATH_INFO: `/v3/projects/d0c12f9c72574a6db10c4d86676e0d03` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:22.184795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c12f9c72574a6db10c4d86676e0d03)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:22.188120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: WARNING py.warnings [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 15:26:22.188120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: warnings.warn(msg) Jul 15 15:26:22.188120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: Jul 15 15:26:22.193483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:22.331944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c12f9c72574a6db10c4d86676e0d03', 'request_id': 'req-f9ef8490-3225-4798-86b3-cb637c4e13c2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.371433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c12f9c72574a6db10c4d86676e0d03', 'request_id': 'req-f9ef8490-3225-4798-86b3-cb637c4e13c2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.390543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c12f9c72574a6db10c4d86676e0d03', 'request_id': 'req-f9ef8490-3225-4798-86b3-cb637c4e13c2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.425358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] The token cache is being invalidate because project d0c12f9c72574a6db10c4d86676e0d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:22.425660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.430685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.436763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 823/1657] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:22 2022] DELETE /identity/v3/projects/d0c12f9c72574a6db10c4d86676e0d03 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:26:22.438080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:22.440292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:22.468315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d87fa-8596-431a-b1ad-b8aebaa729a6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:22.554383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:22.555668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:22.556128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:22.556578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] PATH_INFO: `/v3/users/717eb4a29d41488a8a0937fdcf44d0db` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:22.558330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=717eb4a29d41488a8a0937fdcf44d0db)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:22.562117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:22.643063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717eb4a29d41488a8a0937fdcf44d0db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.655409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717eb4a29d41488a8a0937fdcf44d0db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.670788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717eb4a29d41488a8a0937fdcf44d0db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.672557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717eb4a29d41488a8a0937fdcf44d0db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.673277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717eb4a29d41488a8a0937fdcf44d0db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:22.692905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:22.701110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 835/1658] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:22 2022] DELETE /identity/v3/users/717eb4a29d41488a8a0937fdcf44d0db => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:23.867465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:23.869434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:23.876256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:23.877973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:23.916239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ef8490-3225-4798-86b3-cb637c4e13c2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:24.066805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db36e19-bc2b-43e6-9b9f-17cb6d3b93ff admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:24.081987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:24.084412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:24.085899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:24.087136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] PATH_INFO: `/v3/projects/63772087b05f40a6b22d6f4b8f3d07c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:24.090690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63772087b05f40a6b22d6f4b8f3d07c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:24.096971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:24.113503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:24.115550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:24.116106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:24.116568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] PATH_INFO: `/v3/users/1f963e5fd1474a9fb209410f22e783e8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:24.120445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f963e5fd1474a9fb209410f22e783e8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:24.123070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:24.250304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:24.329369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f963e5fd1474a9fb209410f22e783e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.354279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63772087b05f40a6b22d6f4b8f3d07c5', 'request_id': 'req-fbf1f556-2dcd-4390-96f3-75c673cad7b6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.367264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f963e5fd1474a9fb209410f22e783e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.369327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f963e5fd1474a9fb209410f22e783e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.370349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f963e5fd1474a9fb209410f22e783e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.392637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63772087b05f40a6b22d6f4b8f3d07c5', 'request_id': 'req-fbf1f556-2dcd-4390-96f3-75c673cad7b6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.405465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f7265f50-9b9e-4941-ace1-98f60a2725a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f963e5fd1474a9fb209410f22e783e8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.415349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63772087b05f40a6b22d6f4b8f3d07c5', 'request_id': 'req-fbf1f556-2dcd-4390-96f3-75c673cad7b6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.465220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] The token cache is being invalidate because project 63772087b05f40a6b22d6f4b8f3d07c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:24.465608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.467743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fbf1f556-2dcd-4390-96f3-75c673cad7b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:24.472224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 836/1659] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:23 2022] DELETE /identity/v3/projects/63772087b05f40a6b22d6f4b8f3d07c5 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:24.474461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 824/1660] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:23 2022] DELETE /identity/v3/users/1f963e5fd1474a9fb209410f22e783e8 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:24.480667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:24.480667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:24.480957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:24.481516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4a207-764e-438e-97e6-4003d477c26f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:24.481802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4a207-764e-438e-97e6-4003d477c26f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:24.482092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4a207-764e-438e-97e6-4003d477c26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:24.565519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:24.568772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:24.598675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-88a4a207-764e-438e-97e6-4003d477c26f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:24.608487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-88a4a207-764e-438e-97e6-4003d477c26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:24.820580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 825/1661] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:26:24.851387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 837/1662] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:24.860070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:24.860469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:24.861086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:24.908349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:24.911527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:24.991168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:25.003975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:25.058766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 826/1663] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:25.064310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:25.116778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc55988-641e-4a4f-8a7c-8d3afcda682e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.129656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:25.153037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.153941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.154938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.155346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.158475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:25.160104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.166948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.237553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 827/1664] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:25.239191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:25.257389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbbf99-113a-42f2-a4e5-7aff40ee8681 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.271219 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65b541-a2e6-407d-8618-a131e23e6872 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.275690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.276464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.276744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.277016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.294395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d672b5c-7c4b-4bfa-829d-f16f42a9f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.305802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d672b5c-7c4b-4bfa-829d-f16f42a9f891 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.306176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d672b5c-7c4b-4bfa-829d-f16f42a9f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.306461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d672b5c-7c4b-4bfa-829d-f16f42a9f891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.307273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d672b5c-7c4b-4bfa-829d-f16f42a9f891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.324395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d672b5c-7c4b-4bfa-829d-f16f42a9f891 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.333864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.335632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.391322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 838/1665] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:25.417486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 828/1666] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:25 2022] POST /identity/v3/projects => generated 363 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:25.421424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:25.439115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ee9c4-b755-4eef-b506-631d611f70fe service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.510123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.510957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.511453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.561215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] MFA Rules not processed for user `9a0d49b3b50b4e668e157cb2e8e47da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:25.563414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:25.646923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.647798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.648089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.648348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.649059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.651471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.699547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 839/1667] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:25 2022] POST /identity/v3/users => generated 282 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:25.702143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:25.722987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1c6d4-68e0-4533-817b-9418b43b1216 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.738202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 829/1668] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:25.747508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.748434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.748804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.749262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.750108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.752143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.770471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:25.781817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 840/1669] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:26:25.784834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6f672-4908-4692-a7db-5bdde2c2a896 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.789740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47286695-0686-4bfc-882f-7c07516fbc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.791604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47286695-0686-4bfc-882f-7c07516fbc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.791604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47286695-0686-4bfc-882f-7c07516fbc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.791604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47286695-0686-4bfc-882f-7c07516fbc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.791848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47286695-0686-4bfc-882f-7c07516fbc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.794018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47286695-0686-4bfc-882f-7c07516fbc19 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.807758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 830/1670] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:25.811618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:25.824307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0e94-161f-4740-9f88-00ed0f7744ed admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:25.844472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:25.844832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.845203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.845569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] PATH_INFO: `/v3/projects/c46fa5268abc4514b03e2f2e628af205/users/f55ac2f400054d439e0dd183401b5922/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.885247 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c46fa5268abc4514b03e2f2e628af205, user_id=f55ac2f400054d439e0dd183401b5922, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:25.888507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:25.918200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 841/1671] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:25 2022] PUT /identity/v3/projects/c46fa5268abc4514b03e2f2e628af205/users/f55ac2f400054d439e0dd183401b5922/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:26:25.930173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:25.930482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:25.930770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:25.947905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:25.971874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] MFA Rules not processed for user `f55ac2f400054d439e0dd183401b5922`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:25.974140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:26.061685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 831/1672] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:26:26.063676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:26.164898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b779b3-db31-41dd-bb56-ee4066507f3b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:26.171458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:26.172320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:26.172611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:26.172934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:26.174470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:26.177698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:26.224041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 842/1673] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:26 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:26:26.225681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:26.253895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297a0c2-a1b7-4f10-99b4-b3d0a104db08 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:26.454160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:26.454968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:26.455251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:26.455528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:26.456286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:26.458669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:26.513584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 832/1674] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:26 2022] POST /identity/v3/users => generated 289 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:26.515592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:26.543675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721f813-1337-4a30-8c73-4dd7549e5d69 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:26.553637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:26.554736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:26.555487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:26.555957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:26.556890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:26.560211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:26.571282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:26.586844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 843/1675] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:26.587570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:26.607756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c346bac-82bd-4cf1-8f39-fcbd5eb9a371 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:26.625359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:26.626544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:26.626840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:26.627103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] PATH_INFO: `/v3/projects/08a54815cf144c808b590fd99839f78f/users/a8b6fac03ff24afeb74c3f1ec0ac0ec6/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:26.664168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a54815cf144c808b590fd99839f78f, user_id=a8b6fac03ff24afeb74c3f1ec0ac0ec6, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:26.668815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:26.706095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 833/1676] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:26 2022] PUT /identity/v3/projects/08a54815cf144c808b590fd99839f78f/users/a8b6fac03ff24afeb74c3f1ec0ac0ec6/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:26:26.712618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:26.727711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58e58e-3c17-42f6-9572-50e78bb67ba1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:26.973522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:26.974240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:26.974971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:26.975316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:26.976095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:26.977968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:27.006802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:27.008478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 844/1677] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:27.021441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c584947-dd0d-49f4-a7e8-c3ed1b9d5da6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:27.040908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3b7a0-1d37-40c9-b119-73a58123b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:27.042065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3b7a0-1d37-40c9-b119-73a58123b69f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.045626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3b7a0-1d37-40c9-b119-73a58123b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.046090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3b7a0-1d37-40c9-b119-73a58123b69f admin admin] PATH_INFO: `/v3/projects/08a54815cf144c808b590fd99839f78f/users/a8b6fac03ff24afeb74c3f1ec0ac0ec6/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.057782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3b7a0-1d37-40c9-b119-73a58123b69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a54815cf144c808b590fd99839f78f, user_id=a8b6fac03ff24afeb74c3f1ec0ac0ec6, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:27.064404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3b7a0-1d37-40c9-b119-73a58123b69f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:27.070605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:27.072942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.106716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 834/1678] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:27 2022] PUT /identity/v3/projects/08a54815cf144c808b590fd99839f78f/users/a8b6fac03ff24afeb74c3f1ec0ac0ec6/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:26:27.118753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.119261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.119684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.120560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4655627-96f7-4184-9461-78f7f52450f6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:27.190944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] MFA Rules not processed for user `a8b6fac03ff24afeb74c3f1ec0ac0ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:27.192235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.262976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:27.322202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 835/1679] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:27.337043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:27.339298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.371357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88939f05-0071-4010-af88-b866555158cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:27.372266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939f05-0071-4010-af88-b866555158cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.372553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939f05-0071-4010-af88-b866555158cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.372827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939f05-0071-4010-af88-b866555158cf admin admin] PATH_INFO: `/v3/projects/154d83d3acca49b0b07aceb57c6fa4ed` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.374125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88939f05-0071-4010-af88-b866555158cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=154d83d3acca49b0b07aceb57c6fa4ed)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:27.376127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88939f05-0071-4010-af88-b866555158cf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:27.479681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9d7a1-cc93-4fce-81e8-55b1e5cbb3ba None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:27.502410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-88939f05-0071-4010-af88-b866555158cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154d83d3acca49b0b07aceb57c6fa4ed', 'request_id': 'req-88939f05-0071-4010-af88-b866555158cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:27.516833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-88939f05-0071-4010-af88-b866555158cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154d83d3acca49b0b07aceb57c6fa4ed', 'request_id': 'req-88939f05-0071-4010-af88-b866555158cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:27.542185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-88939f05-0071-4010-af88-b866555158cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154d83d3acca49b0b07aceb57c6fa4ed', 'request_id': 'req-88939f05-0071-4010-af88-b866555158cf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:27.559749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-88939f05-0071-4010-af88-b866555158cf admin admin] The token cache is being invalidate because project 154d83d3acca49b0b07aceb57c6fa4ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:27.560531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-88939f05-0071-4010-af88-b866555158cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:27.561385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-88939f05-0071-4010-af88-b866555158cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:27.567484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 845/1680] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:27 2022] DELETE /identity/v3/projects/154d83d3acca49b0b07aceb57c6fa4ed => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:27.568598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:27.569551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.569719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.570105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.571623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6be9-ec43-407d-89ec-78133bc04adc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.571908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6be9-ec43-407d-89ec-78133bc04adc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.572200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb6be9-ec43-407d-89ec-78133bc04adc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.573872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 846/1681] 10.222.0.40 () {54 vars in 912 bytes} [Fri Jul 15 15:26:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:27.576784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.579943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.580482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.580982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.655421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:27.657411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.714165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:27.716049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ec02-a978-4835-9cc7-344dd880d488 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:27.730137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:27.730570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 836/1682] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:27.732281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7396ab4-61cf-4c91-bd5b-ec0274a312fe None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.732628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7396ab4-61cf-4c91-bd5b-ec0274a312fe None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.733342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7396ab4-61cf-4c91-bd5b-ec0274a312fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.782557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7396ab4-61cf-4c91-bd5b-ec0274a312fe None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:27.784650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7396ab4-61cf-4c91-bd5b-ec0274a312fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.811311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:27.859116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 837/1683] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:27.876557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 847/1684] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:27.881741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.883098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:27.884548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.885460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.886483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.907683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3e63d-77f8-4f6e-aee0-374a4a86612b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:27.919143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:27.920721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:27.922496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:27.923310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.923692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.924154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.933916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 838/1685] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:27.952061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:27.958194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:27.969397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:27.987056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba1d7a-f0ba-4a5d-803f-7836150aadcb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:27.993053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 848/1686] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:27.996001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:27.996783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:27.997279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:27.997627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:27.998445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:27.998837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:28.000805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:28.037373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 839/1687] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:28.042156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:28.059695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ead85a-e1e7-4303-885a-48e690b59947 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:28.065529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:28.066977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.067351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.067716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.071627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:28.077851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:28.205638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 849/1688] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:28 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:28.206168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:28.227270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b5cff-3765-46f5-8192-a730ec6673fa admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:28.461018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:28.465218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.465516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.465800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.473150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:28.473150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:28.526836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:28.527338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 840/1689] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:28 2022] POST /identity/v3/users => generated 285 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:28.557494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68387d-273b-4cbd-ae37-eb33f3882f34 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:28.576755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:28.577454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.577736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.578015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.581831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:28.583537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:28.633797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 850/1690] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:28.636520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:28.662183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7adce-03d9-4e60-b2cc-644207ac61cc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:28.683266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37254c8-6dbd-405d-a797-8b96541950a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:28.684137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37254c8-6dbd-405d-a797-8b96541950a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.684458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37254c8-6dbd-405d-a797-8b96541950a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.684737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37254c8-6dbd-405d-a797-8b96541950a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.690488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37254c8-6dbd-405d-a797-8b96541950a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:28.695452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37254c8-6dbd-405d-a797-8b96541950a2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:28.757339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 841/1691] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:28 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:28.759381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:28.779434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a2b65-6b04-4996-bacb-f0544a56cb8a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:28.792485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847a46a-d91d-43e6-adf8-07a14b72f357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:28.793403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a46a-d91d-43e6-adf8-07a14b72f357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.793682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a46a-d91d-43e6-adf8-07a14b72f357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.793955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847a46a-d91d-43e6-adf8-07a14b72f357 admin admin] PATH_INFO: `/v3/projects/b4db52a8e1264e549d8331d34ae0c903/users/1bf627fcaa8348f1a5bd3e63c7011037/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.847932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847a46a-d91d-43e6-adf8-07a14b72f357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4db52a8e1264e549d8331d34ae0c903, user_id=1bf627fcaa8348f1a5bd3e63c7011037, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:28.851474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847a46a-d91d-43e6-adf8-07a14b72f357 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:28.865898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.866364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.866767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.880032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 851/1692] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:28 2022] PUT /identity/v3/projects/b4db52a8e1264e549d8331d34ae0c903/users/1bf627fcaa8348f1a5bd3e63c7011037/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:28.891251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:28.891546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:28.891824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:28.948583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] MFA Rules not processed for user `08050c1635084492a4b054b172884af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:28.950399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:28.987848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] MFA Rules not processed for user `1bf627fcaa8348f1a5bd3e63c7011037`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:28.991964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:29.127803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 842/1693] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:29.128958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:29.139147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:29.232532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 852/1694] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:29.232532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:29.232532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:29.240663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:29.386011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f454b0e-0337-4466-b89e-fee9b089c003 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:29.399009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:29.400385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:29.400842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:29.401104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] PATH_INFO: `/v3/users/f55ac2f400054d439e0dd183401b5922` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:29.402950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f55ac2f400054d439e0dd183401b5922)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:29.418906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:29.429750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:29.484503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca21de-fde7-44ec-b2b5-075ee080c58c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:29.494626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:29.496152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:29.496328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:29.496692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:29.510384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f55ac2f400054d439e0dd183401b5922'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:29.512227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:29.516850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:29.533454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 853/1695] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:29.534730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:29.536413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:29.547850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f55ac2f400054d439e0dd183401b5922'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:29.548858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f55ac2f400054d439e0dd183401b5922'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:29.549684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f55ac2f400054d439e0dd183401b5922'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:29.564901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f55ac2f400054d439e0dd183401b5922'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:29.600324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 843/1696] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:29 2022] DELETE /identity/v3/users/f55ac2f400054d439e0dd183401b5922 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:29.601847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:29.604192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:29.839067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970c637-b128-45a3-904f-f04eb2994695 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:29.844911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24913acd-079c-4ea0-8b91-297c4922b17f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:29.850574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96df1fd-d326-4733-ab2c-17a04a3a7528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:29.851433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96df1fd-d326-4733-ab2c-17a04a3a7528 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:29.851752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96df1fd-d326-4733-ab2c-17a04a3a7528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:29.852033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96df1fd-d326-4733-ab2c-17a04a3a7528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:29.852743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96df1fd-d326-4733-ab2c-17a04a3a7528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:29.854800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96df1fd-d326-4733-ab2c-17a04a3a7528 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:29.931055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 844/1697] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:29 2022] POST /identity/v3/projects => generated 369 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:29.933574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:29.933866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:29.934143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:29.963607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:29.965115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.005818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:30.084982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:30.087284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.089883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.090284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.094256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.127424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:30.129196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:30.151152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 845/1698] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:30.153490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:30.154667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 854/1699] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:30.155982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:30.157905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.181037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe842889-2bb5-4374-9a38-1148023283c6 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:30.190370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8b53-3e4a-4b0e-9b24-0fdfe0251fc6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:30.190607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:30.191777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.192193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.192674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] PATH_INFO: `/v3/users/5e6af511fedd4788b1b771db772251f3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.195030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6af511fedd4788b1b771db772251f3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:30.197238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:30.205098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:30.206754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.207044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.207320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.208029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:30.216397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:30.306791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 846/1700] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:30 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:30.314183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:30.316992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.350496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6af511fedd4788b1b771db772251f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:30.360242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6af511fedd4788b1b771db772251f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:30.363420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:30.372181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6af511fedd4788b1b771db772251f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:30.374104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6af511fedd4788b1b771db772251f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:30.375226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6af511fedd4788b1b771db772251f3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:30.412801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 855/1701] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:30 2022] DELETE /identity/v3/users/5e6af511fedd4788b1b771db772251f3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:30.414554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:30.419901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.550482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bca20e-bc88-44cc-a18c-3461ea04da73 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:30.664236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:30.664933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.665222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.665503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.674156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.695859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29f0a-6264-4a71-8b90-566732411c8e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:30.706174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e6a56-78cd-4753-b1c4-6e573f7dcaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:30.706817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6a56-78cd-4753-b1c4-6e573f7dcaae admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.707103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6a56-78cd-4753-b1c4-6e573f7dcaae admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.707379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e6a56-78cd-4753-b1c4-6e573f7dcaae admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.708248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e6a56-78cd-4753-b1c4-6e573f7dcaae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:30.712625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e6a56-78cd-4753-b1c4-6e573f7dcaae admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:30.736312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 856/1702] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:30.736312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.736563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.737963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.754421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:30.756260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:30.788407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 847/1703] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:30.789625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:30.816021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:30.816227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4bde-f7c8-4035-9237-0a6da03a32af service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:30.818728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:30.848083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 857/1704] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:30.852915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:30.854903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:30.855191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:30.864302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] PATH_INFO: `/v3/projects/5b54571f194a4637af2f18a695eb5910/users/814bf638133944be9ad10f6ce6b1d5a1/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:30.909609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b54571f194a4637af2f18a695eb5910, user_id=814bf638133944be9ad10f6ce6b1d5a1, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:30.915586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:30.953015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 848/1705] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:30 2022] PUT /identity/v3/projects/5b54571f194a4637af2f18a695eb5910/users/814bf638133944be9ad10f6ce6b1d5a1/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:30.959179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:31.006594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbb459-cfe1-4c47-91a3-10068483a9a2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:31.031802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:31.035745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:31.278638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:31.320870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef70cb-f215-46a3-a4c4-1620e4b507af admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:31.360891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:31.364879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:31.369065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:31.369536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:31.404734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:31.407748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:31.408554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:31.409208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:31.410710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:31.416943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:31.426206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:31.430027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:31.460084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 849/1706] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:26:31.500748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 858/1707] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 543 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:31.503997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:31.531581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e273618-fa74-498a-aab7-f3d25a8bf6d9 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:31.544912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:31.549018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:31.560337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20debb5e-ed19-4721-9db0-e7f199a5dc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:31.561976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debb5e-ed19-4721-9db0-e7f199a5dc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:31.562617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debb5e-ed19-4721-9db0-e7f199a5dc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:31.566784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20debb5e-ed19-4721-9db0-e7f199a5dc2e admin admin] PATH_INFO: `/v3/projects/5b54571f194a4637af2f18a695eb5910/users/814bf638133944be9ad10f6ce6b1d5a1/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:31.570914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debb5e-ed19-4721-9db0-e7f199a5dc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b54571f194a4637af2f18a695eb5910, user_id=814bf638133944be9ad10f6ce6b1d5a1, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:31.577208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20debb5e-ed19-4721-9db0-e7f199a5dc2e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:31.589860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f004df6-ff07-443d-9c80-ecd96fc6755c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:31.600145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 850/1708] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:31 2022] PUT /identity/v3/projects/5b54571f194a4637af2f18a695eb5910/users/814bf638133944be9ad10f6ce6b1d5a1/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:26:31.613031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:31.613498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:31.614345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:31.688745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] MFA Rules not processed for user `814bf638133944be9ad10f6ce6b1d5a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:31.690499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:31.822557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:31.827528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:31.829375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:31.830631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] PATH_INFO: `/v3/projects/c46fa5268abc4514b03e2f2e628af205` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:31.837590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c46fa5268abc4514b03e2f2e628af205)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:31.841110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:31.858932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:31.965401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 851/1709] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:31.980103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:31.981813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:32.045432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c46fa5268abc4514b03e2f2e628af205', 'request_id': 'req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.066619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c46fa5268abc4514b03e2f2e628af205', 'request_id': 'req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.088907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c46fa5268abc4514b03e2f2e628af205', 'request_id': 'req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.116604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] The token cache is being invalidate because project c46fa5268abc4514b03e2f2e628af205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:32.117075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.120767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.126076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 859/1710] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:31 2022] DELETE /identity/v3/projects/c46fa5268abc4514b03e2f2e628af205 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:26:32.128630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:32.131728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:32.256002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3f36-2fad-4179-9357-d1d618b4aac7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:32.347136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5babebf1-339c-43d3-b5ef-ad1a1ae180e5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:32.357877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:32.359119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:32.360316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:32.360785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] PATH_INFO: `/v3/users/a8b6fac03ff24afeb74c3f1ec0ac0ec6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:32.365695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8b6fac03ff24afeb74c3f1ec0ac0ec6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:32.368790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:32.520439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:32.521779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:32.522262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:32.522557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:32.535488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:32.537137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8b6fac03ff24afeb74c3f1ec0ac0ec6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.552350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8b6fac03ff24afeb74c3f1ec0ac0ec6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.567904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8b6fac03ff24afeb74c3f1ec0ac0ec6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.571288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8b6fac03ff24afeb74c3f1ec0ac0ec6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.574300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8b6fac03ff24afeb74c3f1ec0ac0ec6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:32.624328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 860/1711] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:32 2022] DELETE /identity/v3/users/a8b6fac03ff24afeb74c3f1ec0ac0ec6 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:32.637739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:32.717922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:32.720054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08461337-760b-4a23-8441-117edad1b3fe service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:32.750152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 852/1712] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:32.882723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:32.884339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:32.953114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:33.042486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268e06-cd41-4ff7-abf0-13ac2b528095 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:33.057091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:33.058196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:33.058732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:33.059499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:33.064550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:33.195936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:33.198799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee4878-fb81-4c84-b000-6152b7bf9541 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:33.222199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 861/1713] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:34.644970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:34.648101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:34.649072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:34.812204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:34.814821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:35.006910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 853/1714] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:35.213672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:35.214756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:35.215191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:35.255624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:35.259509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:35.292602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] MFA Rules not processed for user `08050c1635084492a4b054b172884af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:35.301856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:35.473510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 862/1715] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:26:35.476490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:35.500022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2344102-5dc0-4f91-b626-02b714751b5a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:35.512766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:35.520471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:35.525010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:35.525333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:35.525624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] PATH_INFO: `/v3/projects/08a54815cf144c808b590fd99839f78f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:35.533858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a54815cf144c808b590fd99839f78f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:35.535892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:35.683433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:35.719203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a54815cf144c808b590fd99839f78f', 'request_id': 'req-f9081bf9-4008-44c3-a467-084b2c432b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:35.737421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a54815cf144c808b590fd99839f78f', 'request_id': 'req-f9081bf9-4008-44c3-a467-084b2c432b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:35.767039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a54815cf144c808b590fd99839f78f', 'request_id': 'req-f9081bf9-4008-44c3-a467-084b2c432b06'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:35.800991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] The token cache is being invalidate because project 08a54815cf144c808b590fd99839f78f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:35.801645 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:35.802172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:35.805524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 854/1716] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:35 2022] DELETE /identity/v3/projects/08a54815cf144c808b590fd99839f78f => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:35.806411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:35.813955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:35.856022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:35.893109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96873e0e-92b3-4440-9560-242a92b4d7f9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:36.043680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9081bf9-4008-44c3-a467-084b2c432b06 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:36.058384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:36.059341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:36.059625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:36.059903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] PATH_INFO: `/v3/users/baad6e8d21fe4d40ab2fc0960438efb0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:36.061465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=baad6e8d21fe4d40ab2fc0960438efb0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:36.063321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:36.139259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:36.141477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:36.141968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:36.144137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:36.150944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:36.156358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baad6e8d21fe4d40ab2fc0960438efb0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:36.175168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baad6e8d21fe4d40ab2fc0960438efb0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:36.175980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baad6e8d21fe4d40ab2fc0960438efb0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:36.176767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baad6e8d21fe4d40ab2fc0960438efb0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:36.192789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-292f3a9f-012e-4652-b35f-7dd0061a95bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baad6e8d21fe4d40ab2fc0960438efb0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:36.244842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 855/1717] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:35 2022] DELETE /identity/v3/users/baad6e8d21fe4d40ab2fc0960438efb0 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:36.246996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d853e-2675-48ba-97e6-d3fbc538a182 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:36.247515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d853e-2675-48ba-97e6-d3fbc538a182 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:36.249306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d853e-2675-48ba-97e6-d3fbc538a182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:36.328336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f68d853e-2675-48ba-97e6-d3fbc538a182 None None] MFA Rules not processed for user `c1a1af21d3f94b70a830b14200619f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:36.329446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f68d853e-2675-48ba-97e6-d3fbc538a182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:36.359160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:36.364051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:36.380549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 863/1718] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 906 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:26:36.404891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:36.406782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:36.423798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fda2b-3b59-44c0-aa67-5019a8c8b0f1 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:36.469820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:36.718376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 856/1719] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:36.719895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cd63e-d7b7-41c7-97d2-a0eee1e067d1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:36.727749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cd63e-d7b7-41c7-97d2-a0eee1e067d1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:36.727749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cd63e-d7b7-41c7-97d2-a0eee1e067d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:36.783943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:36.812123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:36.813254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:36.816315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] PATH_INFO: `/v3/projects/1b2864575fba44f29d22c7afb207a73b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:36.832680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2864575fba44f29d22c7afb207a73b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:36.851178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:36.870616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f49cd63e-d7b7-41c7-97d2-a0eee1e067d1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:36.876277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f49cd63e-d7b7-41c7-97d2-a0eee1e067d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:36.967518 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 857/1720] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:36.972631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cba2a-eceb-4feb-a913-128af51ecca2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:36.982157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cba2a-eceb-4feb-a913-128af51ecca2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:36.984629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126cba2a-eceb-4feb-a913-128af51ecca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:37.085195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-126cba2a-eceb-4feb-a913-128af51ecca2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:37.087860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-126cba2a-eceb-4feb-a913-128af51ecca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:37.098562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:37.121828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2864575fba44f29d22c7afb207a73b', 'request_id': 'req-58a71324-9d34-44e8-ac39-1fc743d797a6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:37.170893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2864575fba44f29d22c7afb207a73b', 'request_id': 'req-58a71324-9d34-44e8-ac39-1fc743d797a6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:37.209633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2864575fba44f29d22c7afb207a73b', 'request_id': 'req-58a71324-9d34-44e8-ac39-1fc743d797a6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:37.244717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] The token cache is being invalidate because project 1b2864575fba44f29d22c7afb207a73b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:37.245256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:37.246753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58a71324-9d34-44e8-ac39-1fc743d797a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:37.250983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 864/1721] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:36 2022] DELETE /identity/v3/projects/1b2864575fba44f29d22c7afb207a73b => generated 0 bytes in 846 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:37.254687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc40f26-acb9-4ba2-9601-3e43be57ca43 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:37.257791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc40f26-acb9-4ba2-9601-3e43be57ca43 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:37.258620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc40f26-acb9-4ba2-9601-3e43be57ca43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:37.326827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ccc40f26-acb9-4ba2-9601-3e43be57ca43 None None] MFA Rules not processed for user `c1a1af21d3f94b70a830b14200619f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:37.330961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ccc40f26-acb9-4ba2-9601-3e43be57ca43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:37.518758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 858/1722] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 550 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:37.522357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:37.523389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:37.523929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:37.568087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 865/1723] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:37.568087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7035f1-a388-4910-95b5-ce7d65e5b841 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:37.568087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7035f1-a388-4910-95b5-ce7d65e5b841 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:37.574686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7035f1-a388-4910-95b5-ce7d65e5b841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:37.615977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cf7035f1-a388-4910-95b5-ce7d65e5b841 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:37.617228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cf7035f1-a388-4910-95b5-ce7d65e5b841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:37.659810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 866/1724] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:37.662566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:37.665529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:37.668124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:37.675148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] MFA Rules not processed for user `2a01407bfc0a480a8f6702993ddb0183`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:37.682947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:37.755513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:37.769887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:37.812063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 867/1725] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:37.812063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:37.815144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:38.002974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d42e0a-c635-41c7-af02-05da9c81cd65 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:38.015987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:38.019489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.020109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.020671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.051485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:38.058485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:38.119005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 868/1726] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:38.124385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:38.128574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:38.208267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 859/1727] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 689 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:38.213206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:38.287947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ea5b0-4d03-4bdb-8a2a-92229726ae3f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:38.348931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:38.349637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.349924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.350203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.351013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:38.351324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:38.353131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ac388-ae5e-4112-9967-b8550d4fc276 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:38.444311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 860/1728] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:38.444311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675584-25a9-47cc-b456-e683a6634fda None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.444311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675584-25a9-47cc-b456-e683a6634fda None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.444311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1675584-25a9-47cc-b456-e683a6634fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.520354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c1675584-25a9-47cc-b456-e683a6634fda None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:38.524868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c1675584-25a9-47cc-b456-e683a6634fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:38.552350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:38.570851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 861/1729] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:38.572668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.572976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.573255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.652457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36474a-247f-4afa-97f6-6f3565ef95e0 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:38.702588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:38.711130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.711636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.712159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.729805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] MFA Rules not processed for user `2a01407bfc0a480a8f6702993ddb0183`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:38.732427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:38.738021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:38.807154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 862/1730] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:38.815707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:38.819688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:38.821435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:38.864084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 869/1731] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 744 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:38.865818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:38.912430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f0e0e-88a3-499a-97b0-365333c3dc99 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:38.928710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a607b4-3e3b-45a1-9bd1-a2e188b7f546 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:38.946317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:38.952022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.952400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.962339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:38.966453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:38.966939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.967570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:38.968169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:38.971569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:38.975616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:38.980780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:38.981585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:38.993137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:39.064112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 870/1732] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:39.069400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:39.085741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 863/1733] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:38 2022] POST /identity/v3/projects => generated 387 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:39.089326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:39.108893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9074038-e5dd-4905-ab20-3ef2c935ae64 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:39.121314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:39.374988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc5fb0-f2b9-4ab9-a02a-6bae3bb78e94 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:39.401993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:39.404193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:39.404810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:39.405764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:39.420618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:39.426057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:39.507018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 864/1734] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:39 2022] POST /identity/v3/users => generated 294 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:39.531303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:39.570875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:39.573415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:39.574722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:39.576984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:39.580061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:39.589823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:39.621790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67177c-7403-4800-a9d4-bd1161deece7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:39.647605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:39.648734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:39.659736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:39.659736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:39.683322 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:39.689779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:39.718209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 865/1735] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:39.740349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:39.758562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 871/1736] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:39 2022] POST /identity/v3/projects => generated 405 bytes in 690 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:39.764138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:39.808531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09f6b9-813a-4738-aa6f-4c5c51f952c4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:40.260787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:40.261494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:40.261780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:40.262058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:40.266145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:40.274943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:40.281583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd9e5c-9dd6-4964-9e25-d24442da51d4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:40.305464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 866/1737] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:39 2022] GET /identity/v3/roles => generated 1420 bytes in 583 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:40.309476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:40.315603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:40.320196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:40.321726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:40.322424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:40.323217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:40.325057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:40.329441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:40.452135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 872/1738] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:39 2022] POST /identity/v3/users => generated 303 bytes in 688 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:40.465786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:40.528898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd957-870f-45d3-b8c0-41f489b7f439 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:40.596336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:40.597372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:40.597793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:40.606064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] PATH_INFO: `/v3/projects/4cd3f97e24a64e019cdd532661813046/users/3e7bc08c237f46d3808eecf20a64463a/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:40.708057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd3f97e24a64e019cdd532661813046, user_id=3e7bc08c237f46d3808eecf20a64463a, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:40.711509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:40.768900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 873/1739] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:40 2022] PUT /identity/v3/projects/4cd3f97e24a64e019cdd532661813046/users/3e7bc08c237f46d3808eecf20a64463a/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:40.770682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:40.781362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b84cb0-b484-4773-95f9-9a5f4e0bf419 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:40.850327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:40.917446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:40.995971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506d38f-0352-4ef6-b28b-7c596f185362 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:41.030511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2e0bf-8a2d-43b3-9b59-bc9666a9e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:41.031995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2e0bf-8a2d-43b3-9b59-bc9666a9e708 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.039274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2e0bf-8a2d-43b3-9b59-bc9666a9e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.039946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2e0bf-8a2d-43b3-9b59-bc9666a9e708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.041504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2e0bf-8a2d-43b3-9b59-bc9666a9e708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:41.044920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee2e0bf-8a2d-43b3-9b59-bc9666a9e708 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:41.078847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 874/1740] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:41.083511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5458a32-2e8c-40de-b092-4291ac417be2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.084232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5458a32-2e8c-40de-b092-4291ac417be2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.088167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5458a32-2e8c-40de-b092-4291ac417be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.177738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e5458a32-2e8c-40de-b092-4291ac417be2 None None] MFA Rules not processed for user `3e7bc08c237f46d3808eecf20a64463a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:41.191675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5458a32-2e8c-40de-b092-4291ac417be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:41.237352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:41.242005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.242556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.243044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.566071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 875/1741] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:41.568504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85cde2-2aa7-4b17-81fa-69341ef8d9af None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.570475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85cde2-2aa7-4b17-81fa-69341ef8d9af None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.571036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85cde2-2aa7-4b17-81fa-69341ef8d9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.602853 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:41.609294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:41.649793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 867/1742] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 1343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:41.652078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:41.659600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6b85cde2-2aa7-4b17-81fa-69341ef8d9af None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:41.662367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b85cde2-2aa7-4b17-81fa-69341ef8d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:41.703955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766a52-6e66-425d-a89c-76ce1ed817a9 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:41.728062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:41.729409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.729897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.730367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.739034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:41.742465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:41.789130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 868/1743] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:41.791284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:41.802056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:41.876605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf7b34-fe66-4a09-80d6-f4d4baffcf76 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:41.897082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:41.900536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.901177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.901463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] PATH_INFO: `/v3/users/1bf627fcaa8348f1a5bd3e63c7011037` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.905332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf627fcaa8348f1a5bd3e63c7011037)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:41.909279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:41.948446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 876/1744] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Jul 15 15:26:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:41.977347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f6a2b-c379-433c-b9cb-ac1f5f556c7a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:41.977994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f6a2b-c379-433c-b9cb-ac1f5f556c7a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:41.987213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f6a2b-c379-433c-b9cb-ac1f5f556c7a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:41.990630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 877/1745] 10.222.0.40 () {54 vars in 912 bytes} [Fri Jul 15 15:26:41 2022] GET /identity => generated 270 bytes in 39 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:42.067673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f6a2b-c379-433c-b9cb-ac1f5f556c7a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:42.160385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf627fcaa8348f1a5bd3e63c7011037'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:42.192395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf627fcaa8348f1a5bd3e63c7011037'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:42.194964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf627fcaa8348f1a5bd3e63c7011037'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:42.198866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf627fcaa8348f1a5bd3e63c7011037'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:42.238069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f6a2b-c379-433c-b9cb-ac1f5f556c7a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:42.239319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf627fcaa8348f1a5bd3e63c7011037'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:42.261205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:42.262445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:42.262985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:42.263735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:42.265974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:42.280888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:42.296237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 869/1746] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:41 2022] DELETE /identity/v3/users/1bf627fcaa8348f1a5bd3e63c7011037 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:42.296237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:42.303279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:42.407841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 878/1747] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:42 2022] POST /identity/v3/projects => generated 385 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:42.409906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:42.421012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:42.433433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:42.558126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13341753-e810-4429-b44d-d1b3721ad954 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:42.601861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:42.677849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fce93-b2fd-4f42-81e6-1882aa4f86f3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:42.685780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:42.686892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:42.687357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:42.687797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:42.690259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:42.705971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:42.708257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1734e-ebae-45bd-90e2-f71050c28b2e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:42.720170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 879/1748] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:42.723919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:42.724520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:42.725450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:42.728162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45226d6a-469d-4a74-a77c-ff3bbe0e0a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:42.729284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45226d6a-469d-4a74-a77c-ff3bbe0e0a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:42.729569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45226d6a-469d-4a74-a77c-ff3bbe0e0a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:42.729882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45226d6a-469d-4a74-a77c-ff3bbe0e0a29 admin admin] PATH_INFO: `/v3/projects/cbc1379bc4d64b6eb34ddc942aaef79a/users/ba226c21d10b4e74a602ebb859372814/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:42.774033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:42.776730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45226d6a-469d-4a74-a77c-ff3bbe0e0a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc1379bc4d64b6eb34ddc942aaef79a, user_id=ba226c21d10b4e74a602ebb859372814, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:42.778367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:42.780184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45226d6a-469d-4a74-a77c-ff3bbe0e0a29 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:42.810601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 880/1749] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:42.813402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:42.818393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:42.821266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 870/1750] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:42 2022] PUT /identity/v3/projects/cbc1379bc4d64b6eb34ddc942aaef79a/users/ba226c21d10b4e74a602ebb859372814/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:42.822661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857803-cfaa-4e9d-b30b-15c8e9e43024 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:42.822954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857803-cfaa-4e9d-b30b-15c8e9e43024 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:42.823241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40857803-cfaa-4e9d-b30b-15c8e9e43024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:42.880298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a45ae1-9948-413d-984b-3786a8c7dce5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:42.935041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-40857803-cfaa-4e9d-b30b-15c8e9e43024 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:42.941723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-40857803-cfaa-4e9d-b30b-15c8e9e43024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:43.082212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:43.085919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:43.086207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:43.086493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] PATH_INFO: `/v3/projects/c1ed18ddda3846ad971faedc3cd6dfb7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:43.091644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ed18ddda3846ad971faedc3cd6dfb7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:43.097039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:43.258560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ed18ddda3846ad971faedc3cd6dfb7', 'request_id': 'req-874daa58-aa1a-4105-a692-79922225d8eb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:43.282510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ed18ddda3846ad971faedc3cd6dfb7', 'request_id': 'req-874daa58-aa1a-4105-a692-79922225d8eb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:43.308995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ed18ddda3846ad971faedc3cd6dfb7', 'request_id': 'req-874daa58-aa1a-4105-a692-79922225d8eb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:43.344686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] The token cache is being invalidate because project c1ed18ddda3846ad971faedc3cd6dfb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:43.344686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:43.347458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:43.371277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 881/1752] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:42 2022] DELETE /identity/v3/projects/c1ed18ddda3846ad971faedc3cd6dfb7 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:43.374427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 871/1752] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 549 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:43.378267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:43.379017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:43.379495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:43.383155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:43.387022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:43.569731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] MFA Rules not processed for user `ba226c21d10b4e74a602ebb859372814`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:43.569731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:43.659674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874daa58-aa1a-4105-a692-79922225d8eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:43.703948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:43.703948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:43.703948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:43.703948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:43.704950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:43.726342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:43.760111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:43.808543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 882/1753] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:43 2022] POST /identity/v3/users => generated 299 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:43.809528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:43.812272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:43.852573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:44.061893 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 872/1754] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 687 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:44.064428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:44.080061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:44.203551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a17ebf8-b3bd-4357-9525-f85c1845ebfa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:44.207277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d98a29-6940-4f89-9c8b-9b298b0c8cc1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:44.232710 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:44.236876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:44.247817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:44.248918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:44.250555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:44.251298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:44.251786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] PATH_INFO: `/v3/users/eeb56e922e33492bb8d04e68b607a7cd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:44.256843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb56e922e33492bb8d04e68b607a7cd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:44.260693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:44.276407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:44.291905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:44.382094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:44.385090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:44.416180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb56e922e33492bb8d04e68b607a7cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:44.465609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 883/1755] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:44.466400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb56e922e33492bb8d04e68b607a7cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:44.473742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:44.475403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb56e922e33492bb8d04e68b607a7cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:44.476738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb56e922e33492bb8d04e68b607a7cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:44.510495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb56e922e33492bb8d04e68b607a7cd'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:44.541149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0b0bd-1cd8-4e90-9807-57a8ef13a3a7 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:44.567883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:44.569367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:44.571035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:44.572092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:44.578228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:44.615204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:44.617229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:44.637351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 884/1756] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:44.643304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:44.650351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:44.676725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 873/1757] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:44 2022] DELETE /identity/v3/users/eeb56e922e33492bb8d04e68b607a7cd => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:44.676725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:44.679446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:44.735764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc116fbc-691d-4a83-965b-4a51c32da06b service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:45.051834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c5fd9-2f8c-4b50-b6a9-86b5c67413ad admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:45.074498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b7f65-070c-4ef8-ad51-2a7c08aaec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:45.075217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b7f65-070c-4ef8-ad51-2a7c08aaec25 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:45.075855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b7f65-070c-4ef8-ad51-2a7c08aaec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:45.077140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b7f65-070c-4ef8-ad51-2a7c08aaec25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:45.081185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b7f65-070c-4ef8-ad51-2a7c08aaec25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:45.086909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b7f65-070c-4ef8-ad51-2a7c08aaec25 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:45.095962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:45.099592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:45.106934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:45.107681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:45.109042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:45.115054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:45.140367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 885/1758] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:45.140367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e43d8f-48a3-4b09-b0c9-a0850dd5322d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:45.140367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e43d8f-48a3-4b09-b0c9-a0850dd5322d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:45.140367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e43d8f-48a3-4b09-b0c9-a0850dd5322d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:45.141043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 886/1759] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:26:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:26:45.141470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e43d8f-48a3-4b09-b0c9-a0850dd5322d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:45.214258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e43d8f-48a3-4b09-b0c9-a0850dd5322d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:45.247473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:45.248994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:45.249483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:45.270415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] PATH_INFO: `/v3/projects/b509253e26f7468d8787fc25ad55dbf4/users/a4e6da7145534b5b87edb4ea9ca83372/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:45.303614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 874/1760] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:44 2022] POST /identity/v3/projects => generated 403 bytes in 631 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:45.315841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:45.391830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b509253e26f7468d8787fc25ad55dbf4, user_id=a4e6da7145534b5b87edb4ea9ca83372, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:45.427609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:45.505805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 887/1761] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:45 2022] PUT /identity/v3/projects/b509253e26f7468d8787fc25ad55dbf4/users/a4e6da7145534b5b87edb4ea9ca83372/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:45.523723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:45.650237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:45.738638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe4144-7777-4c49-9fef-0f666057c93a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:45.907149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:46.031621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.036280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.036590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.036880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.038989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.045352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:46.060430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb950ea-1af7-40cb-b98b-43aede7a30c1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:46.073040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacd6eb-8935-4f6e-89f5-82d13cadc0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.074736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd6eb-8935-4f6e-89f5-82d13cadc0da admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.075157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd6eb-8935-4f6e-89f5-82d13cadc0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.076012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacd6eb-8935-4f6e-89f5-82d13cadc0da admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.077641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacd6eb-8935-4f6e-89f5-82d13cadc0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.079576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacd6eb-8935-4f6e-89f5-82d13cadc0da admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:46.101170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 888/1762] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 578 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:46.103905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 875/1763] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:45 2022] POST /identity/v3/users => generated 308 bytes in 796 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:46.104903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.105561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.106205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.107207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:46.136003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7c7e7-2014-4844-9a33-315309ece28e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:46.148797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.150135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.151704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.152244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:46.154208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.154874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:46.156679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.166346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:46.191824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 876/1764] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:46.196814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:46.222487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87043e-f44b-4b0e-8409-4b32ed4b1599 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:46.248287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101ba6b-d463-4631-8a4b-37bba77316c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.250636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ba6b-d463-4631-8a4b-37bba77316c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.252372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ba6b-d463-4631-8a4b-37bba77316c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.252996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101ba6b-d463-4631-8a4b-37bba77316c4 admin admin] PATH_INFO: `/v3/projects/b509253e26f7468d8787fc25ad55dbf4/users/a4e6da7145534b5b87edb4ea9ca83372/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.264913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ba6b-d463-4631-8a4b-37bba77316c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b509253e26f7468d8787fc25ad55dbf4, user_id=a4e6da7145534b5b87edb4ea9ca83372, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.292116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101ba6b-d463-4631-8a4b-37bba77316c4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:46.341344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 889/1765] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:46.367495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:46.397470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01161f-1363-40dc-a66c-96917a95c003 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:46.413067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 877/1766] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:46 2022] PUT /identity/v3/projects/b509253e26f7468d8787fc25ad55dbf4/users/a4e6da7145534b5b87edb4ea9ca83372/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:46.425456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.426170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.426356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.522969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.523379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] MFA Rules not processed for user `a4e6da7145534b5b87edb4ea9ca83372`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:46.524513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.525050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.525164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:46.526148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] PATH_INFO: `/v3/projects/6e48ad659b364fb09010d3b81051bb7a/users/27a3ccc8b8bc494191ce9629ad4ac242/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.546548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e48ad659b364fb09010d3b81051bb7a, user_id=27a3ccc8b8bc494191ce9629ad4ac242, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.550259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:46.579913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 890/1767] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:46 2022] PUT /identity/v3/projects/6e48ad659b364fb09010d3b81051bb7a/users/27a3ccc8b8bc494191ce9629ad4ac242/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:46.586314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:46.770336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e52aa3-3c64-4467-bd92-3aff6aaf081f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:46.784404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.785747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.787946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.788519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.790207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.798506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:46.821445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:46.822354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 878/1768] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:46.839178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:46.843559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:46.850285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 891/1769] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:46.852880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:46.880058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3b970-6af9-49e0-9cff-16c255797761 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:46.918687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49697f2b-88f3-4f45-99eb-94115dae07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:46.921654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49697f2b-88f3-4f45-99eb-94115dae07c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:46.923362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49697f2b-88f3-4f45-99eb-94115dae07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:46.926103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49697f2b-88f3-4f45-99eb-94115dae07c5 admin admin] PATH_INFO: `/v3/projects/6e48ad659b364fb09010d3b81051bb7a/users/27a3ccc8b8bc494191ce9629ad4ac242/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:46.938685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49697f2b-88f3-4f45-99eb-94115dae07c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e48ad659b364fb09010d3b81051bb7a, user_id=27a3ccc8b8bc494191ce9629ad4ac242, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:46.953768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49697f2b-88f3-4f45-99eb-94115dae07c5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:47.007104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:47.019479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 892/1770] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:46 2022] PUT /identity/v3/projects/6e48ad659b364fb09010d3b81051bb7a/users/27a3ccc8b8bc494191ce9629ad4ac242/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:47.032962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:47.033437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:47.033900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:47.092971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45648b6a-00e1-4ede-8a17-35fd67fda283 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:47.127794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] MFA Rules not processed for user `27a3ccc8b8bc494191ce9629ad4ac242`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:47.129495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:47.345132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 893/1771] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:47 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:26:47.371729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:47.374370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:47.475169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:47.476367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:47.476972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:47.477477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:47.516797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9737ae-aa8f-461c-8faa-d68823eb039b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:47.541892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:47.543105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:47.543592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:47.545022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:47.614241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:47.643191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:47.736029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 894/1772] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:47.945650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:47.948874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:47.995595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 879/1773] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 1157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:48.139715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:48.266294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:48.268567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:48.379059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea999065-3f7d-49c5-9113-24215aa7abed service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:48.387525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:48.389109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:48.389580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:48.390037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] PATH_INFO: `/v3/users/ba226c21d10b4e74a602ebb859372814` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:48.392607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba226c21d10b4e74a602ebb859372814)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:48.397483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:48.425226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:48.529351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba226c21d10b4e74a602ebb859372814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:48.552493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba226c21d10b4e74a602ebb859372814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:48.576294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba226c21d10b4e74a602ebb859372814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:48.576934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba226c21d10b4e74a602ebb859372814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:48.577717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba226c21d10b4e74a602ebb859372814'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:48.617239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 895/1774] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:48 2022] DELETE /identity/v3/users/ba226c21d10b4e74a602ebb859372814 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:48.618227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:48.622505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:48.755378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b785eec-80a3-4adb-91b0-33b7e563b92f service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:48.758766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1b592-cdc8-4064-9c85-942d3897606a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:48.771398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:48.772375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:48.772658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:48.772934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] PATH_INFO: `/v3/users/3e7bc08c237f46d3808eecf20a64463a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:48.776818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7bc08c237f46d3808eecf20a64463a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:48.786057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:48.969084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:49.104046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7bc08c237f46d3808eecf20a64463a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.111158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:49.117408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:49.122492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:49.124613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:49.128751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7bc08c237f46d3808eecf20a64463a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.143468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:49.160309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7bc08c237f46d3808eecf20a64463a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.160309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7bc08c237f46d3808eecf20a64463a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.164562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7bc08c237f46d3808eecf20a64463a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.237503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:49.257590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec90f1e-c576-43aa-9255-01f86f72ce13 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:49.272453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 896/1775] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:48 2022] DELETE /identity/v3/users/3e7bc08c237f46d3808eecf20a64463a => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:49.273511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:49.276061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:49.330546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:49.343474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 880/1776] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 1078 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:26:49.351431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5696c-15df-4a22-89ff-d4c18d24adc9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:49.351888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5696c-15df-4a22-89ff-d4c18d24adc9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:49.352339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5696c-15df-4a22-89ff-d4c18d24adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:49.442181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-aea5696c-15df-4a22-89ff-d4c18d24adc9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:49.450965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-aea5696c-15df-4a22-89ff-d4c18d24adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:49.618874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba1ab2-51c3-4b50-a781-529d982664de admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:49.631705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:49.633706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:49.634170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:49.635370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] PATH_INFO: `/v3/projects/b4db52a8e1264e549d8331d34ae0c903` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:49.637501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4db52a8e1264e549d8331d34ae0c903)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:49.640483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:49.824779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 881/1777] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:49.838429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:49.843655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:49.848265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:49.879366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4db52a8e1264e549d8331d34ae0c903', 'request_id': 'req-401dd117-d720-41f5-98ba-f89bc8de2f23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.900991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4db52a8e1264e549d8331d34ae0c903', 'request_id': 'req-401dd117-d720-41f5-98ba-f89bc8de2f23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.913335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:49.915531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4db52a8e1264e549d8331d34ae0c903', 'request_id': 'req-401dd117-d720-41f5-98ba-f89bc8de2f23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.916653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:49.977295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] The token cache is being invalidate because project b4db52a8e1264e549d8331d34ae0c903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:49.977578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.979878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:49.988741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 897/1778] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:49 2022] DELETE /identity/v3/projects/b4db52a8e1264e549d8331d34ae0c903 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:49.990381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:49.996524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:50.252746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 882/1779] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:50.259863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:50.268796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:50.309354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c030c-3846-4263-b1a3-e298b16f878a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:50.329199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:50.330058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:50.330343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:50.330715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] PATH_INFO: `/v3/users/814bf638133944be9ad10f6ce6b1d5a1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:50.332279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=814bf638133944be9ad10f6ce6b1d5a1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:50.334347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:50.404948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd117-d720-41f5-98ba-f89bc8de2f23 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:50.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:50.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:50.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:50.446947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] PATH_INFO: `/v3/projects/3e116f54062843169dbffbfa130584a0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:50.447727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e116f54062843169dbffbfa130584a0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:50.447727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:50.512258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814bf638133944be9ad10f6ce6b1d5a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.542773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814bf638133944be9ad10f6ce6b1d5a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.549636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814bf638133944be9ad10f6ce6b1d5a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.553932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814bf638133944be9ad10f6ce6b1d5a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.587912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814bf638133944be9ad10f6ce6b1d5a1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.591036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:50.645814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 883/1780] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:50 2022] DELETE /identity/v3/users/814bf638133944be9ad10f6ce6b1d5a1 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:50.653815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:50.659059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:50.667620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:50.683104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e116f54062843169dbffbfa130584a0', 'request_id': 'req-8c83409d-150c-46fc-938d-f22231eb55cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.699729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e116f54062843169dbffbfa130584a0', 'request_id': 'req-8c83409d-150c-46fc-938d-f22231eb55cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.728450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e116f54062843169dbffbfa130584a0', 'request_id': 'req-8c83409d-150c-46fc-938d-f22231eb55cc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.769928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] The token cache is being invalidate because project 3e116f54062843169dbffbfa130584a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:50.770604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.772811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c83409d-150c-46fc-938d-f22231eb55cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:50.798214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 898/1781] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:49 2022] DELETE /identity/v3/projects/3e116f54062843169dbffbfa130584a0 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:50.809658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4c799-6c5f-4169-b95d-2ea95857a55c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:50.810838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4c799-6c5f-4169-b95d-2ea95857a55c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:50.811562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4c799-6c5f-4169-b95d-2ea95857a55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:50.855248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4c15c-cf2a-4868-b198-72e5c2484783 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:50.866239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:50.866955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:50.867236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:50.867513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:50.872077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:50.888704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fdc4c799-6c5f-4169-b95d-2ea95857a55c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:50.891797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fdc4c799-6c5f-4169-b95d-2ea95857a55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:51.047981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:51.057181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:51.138604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 884/1782] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:51.147201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:51.153297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:51.189450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 899/1783] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:51.197378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:51.199139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:51.201265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:51.223740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e050686-d0d9-416a-9a6c-fb35fa1f27fa service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:51.262173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:51.267041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:51.267542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:51.268029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:51.278406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:51.297400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:51.299094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:51.311061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 900/1784] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:51.326324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:51.327994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:51.332732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:51.378711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 885/1785] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:51.386362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27b862-0899-4750-84cb-96e7765658e7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:51.412695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:51.416851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:51.420554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:51.421276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:51.422738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:51.424385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:51.426546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:51.465313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 901/1786] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:51.467513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:51.484137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216ac52-fbc5-439e-a563-c5331cedf006 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:51.497639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:51.504551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:51.504922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:51.505512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:51.508937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:51.511262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:51.632390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 886/1787] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:51 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:26:51.632699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:51.657524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e54a5-69de-48f7-b9c0-c4cc2399fb92 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:51.931191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:51.932182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:51.932530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:51.932819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:51.933828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:51.954049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:51.997942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 902/1788] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:51 2022] POST /identity/v3/users => generated 289 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:26:51.999631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:52.029593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd3092-1c62-45ce-9c9e-bc530465e640 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:52.043319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:52.044457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:52.044955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:52.045429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:52.046834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:52.055595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:52.096533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 887/1789] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:52.099708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:52.205043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33827987-5fe7-4bae-9908-73808ce9eb82 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:52.221411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598f678-2b90-415f-820b-fe96a65331ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:52.223418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f678-2b90-415f-820b-fe96a65331ec admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:52.223647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f678-2b90-415f-820b-fe96a65331ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:52.223876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598f678-2b90-415f-820b-fe96a65331ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:52.224634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f678-2b90-415f-820b-fe96a65331ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:52.226393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598f678-2b90-415f-820b-fe96a65331ec admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:52.256432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 903/1790] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:52.263522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:52.297970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76f95e-be89-4a74-845e-a2c93dd936d0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:52.307081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377b0fe-e189-49b4-bd6d-04bfb19fa0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:52.309723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377b0fe-e189-49b4-bd6d-04bfb19fa0f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:52.310563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377b0fe-e189-49b4-bd6d-04bfb19fa0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:52.311079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377b0fe-e189-49b4-bd6d-04bfb19fa0f5 admin admin] PATH_INFO: `/v3/projects/e3b59a0d413f4edd8e465e2f365fa6f0/users/51d737907e404f28a6770f588fe7c651/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:52.355532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377b0fe-e189-49b4-bd6d-04bfb19fa0f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b59a0d413f4edd8e465e2f365fa6f0, user_id=51d737907e404f28a6770f588fe7c651, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:52.382644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377b0fe-e189-49b4-bd6d-04bfb19fa0f5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:52.429899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 888/1791] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:52 2022] PUT /identity/v3/projects/e3b59a0d413f4edd8e465e2f365fa6f0/users/51d737907e404f28a6770f588fe7c651/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:52.454014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:52.454726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:52.455256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:52.503094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:52.506609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:52.507344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:52.516206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] MFA Rules not processed for user `51d737907e404f28a6770f588fe7c651`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:52.520337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:52.570215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] MFA Rules not processed for user `c1a1af21d3f94b70a830b14200619f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:52.584906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:52.602425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:52.638214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:52.768769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 889/1792] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:52.777714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 904/1793] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:52.782210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:52.991293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18f4d-c28b-4578-8f60-0b83c3f4b939 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:53.010071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:53.011418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:53.012083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:53.012569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:53.013899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:53.017477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:53.108697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 890/1794] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:52 2022] POST /identity/v3/projects => generated 377 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:53.111016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:53.178988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081dc50f-00b1-4a02-b50a-70676357c091 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:53.349592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:53.350770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:53.351258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:53.351578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:53.352664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:53.372236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:53.437717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 905/1795] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:53 2022] POST /identity/v3/users => generated 295 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:53.439889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:53.472652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6bcc4-0a39-43e5-859d-6b73b5922576 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:53.490968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:53.491675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:53.492138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:53.492302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:53.493107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:53.494934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:53.527866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 891/1796] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:53.530455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:53.578344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560d59c-a23c-4c10-a927-65699704e0b5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:53.593821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:53.595648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:53.596148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:53.596584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] PATH_INFO: `/v3/projects/c3f4526f78514a67a42403a281f384da/users/1c62c41ab7ff4072b4132cf66f62b22a/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:53.661816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f4526f78514a67a42403a281f384da, user_id=1c62c41ab7ff4072b4132cf66f62b22a, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:53.665380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:53.695624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 906/1797] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:53 2022] PUT /identity/v3/projects/c3f4526f78514a67a42403a281f384da/users/1c62c41ab7ff4072b4132cf66f62b22a/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:53.701410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:53.725531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4070b-b8db-402f-b838-04291557bd25 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:53.919381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9668dc-a1d4-438b-b6f6-b44de7c7e7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:53.922454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9668dc-a1d4-438b-b6f6-b44de7c7e7ba admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:53.924888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9668dc-a1d4-438b-b6f6-b44de7c7e7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:53.926150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9668dc-a1d4-438b-b6f6-b44de7c7e7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:53.928623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9668dc-a1d4-438b-b6f6-b44de7c7e7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:53.930146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9668dc-a1d4-438b-b6f6-b44de7c7e7ba admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:53.967532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:53.985498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead76888-816e-4b5c-871d-1651c6a76a6f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:53.991275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 892/1798] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:54.013845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:54.015010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:54.015301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:54.015594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] PATH_INFO: `/v3/projects/c3f4526f78514a67a42403a281f384da/users/1c62c41ab7ff4072b4132cf66f62b22a/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:54.025136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f4526f78514a67a42403a281f384da, user_id=1c62c41ab7ff4072b4132cf66f62b22a, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:54.028743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:54.081905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 907/1799] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:53 2022] PUT /identity/v3/projects/c3f4526f78514a67a42403a281f384da/users/1c62c41ab7ff4072b4132cf66f62b22a/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:54.092877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:54.093758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:54.093758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:54.203690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] MFA Rules not processed for user `1c62c41ab7ff4072b4132cf66f62b22a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:54.208332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:54.254273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:54.260722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:54.335270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:54.546823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 893/1800] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:54 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:54.548262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:54.550230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:54.562709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a11265-1655-4715-824f-6246e22c9906 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:54.588830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76958f5b-f5f4-4a96-b967-390e22ae6f56 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:54.595429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:54.597538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:54.598096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:54.598650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] PATH_INFO: `/v3/projects/cbc1379bc4d64b6eb34ddc942aaef79a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:54.600759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:54.601757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc1379bc4d64b6eb34ddc942aaef79a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:54.602399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:54.606288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:54.606588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:54.607112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] PATH_INFO: `/v3/projects/4cd3f97e24a64e019cdd532661813046` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:54.610143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd3f97e24a64e019cdd532661813046)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:54.614188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:54.691255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:54.796911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd3f97e24a64e019cdd532661813046', 'request_id': 'req-249c74d2-20b7-499a-8f1d-e69c89f6fb07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.821400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd3f97e24a64e019cdd532661813046', 'request_id': 'req-249c74d2-20b7-499a-8f1d-e69c89f6fb07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.858750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd3f97e24a64e019cdd532661813046', 'request_id': 'req-249c74d2-20b7-499a-8f1d-e69c89f6fb07'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.860579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc1379bc4d64b6eb34ddc942aaef79a', 'request_id': 'req-184de27f-4c6c-4158-b800-6a2be7413255'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.890607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc1379bc4d64b6eb34ddc942aaef79a', 'request_id': 'req-184de27f-4c6c-4158-b800-6a2be7413255'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.917586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] The token cache is being invalidate because project 4cd3f97e24a64e019cdd532661813046 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:54.917886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.918779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.920891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 894/1801] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:54 2022] DELETE /identity/v3/projects/4cd3f97e24a64e019cdd532661813046 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:54.923552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc1379bc4d64b6eb34ddc942aaef79a', 'request_id': 'req-184de27f-4c6c-4158-b800-6a2be7413255'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.926566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:54.934247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:54.996216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] The token cache is being invalidate because project cbc1379bc4d64b6eb34ddc942aaef79a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:54.996624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:54.999708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:55.016917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 908/1802] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:54 2022] DELETE /identity/v3/projects/cbc1379bc4d64b6eb34ddc942aaef79a => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:55.016917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:55.024343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:55.203066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c74d2-20b7-499a-8f1d-e69c89f6fb07 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:55.258264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184de27f-4c6c-4158-b800-6a2be7413255 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:55.280377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:55.283956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:55.286929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:55.287509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] PATH_INFO: `/v3/projects/5b54571f194a4637af2f18a695eb5910` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:55.303740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b54571f194a4637af2f18a695eb5910)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:55.316270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:55.397240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:55.398061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:55.398493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:55.399050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:55.403385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:55.485226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:55.618942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b54571f194a4637af2f18a695eb5910', 'request_id': 'req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:55.685639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b54571f194a4637af2f18a695eb5910', 'request_id': 'req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:55.750391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b54571f194a4637af2f18a695eb5910', 'request_id': 'req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:55.789146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:55.791000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:55.804232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 895/1803] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 883 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:55.805200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:55.806849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:55.845560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73828f78-020f-4bc8-a81b-8e7e75f106e0 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:55.851136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] The token cache is being invalidate because project 5b54571f194a4637af2f18a695eb5910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:55.855240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:55.869128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:55.876288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 909/1804] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:55 2022] DELETE /identity/v3/projects/5b54571f194a4637af2f18a695eb5910 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:55.878055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:55.890802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:56.132332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:56.174398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:56.179932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:56.182492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:56.184643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] PATH_INFO: `/v3/users/a4e6da7145534b5b87edb4ea9ca83372` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:56.186931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e6da7145534b5b87edb4ea9ca83372)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:56.192604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:56.227614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b593d-924e-4ba0-b534-ebe715c3d7ad admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:56.236033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:56.237712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:56.238401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:56.239492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] PATH_INFO: `/v3/users/27a3ccc8b8bc494191ce9629ad4ac242` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:56.242017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a3ccc8b8bc494191ce9629ad4ac242)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:56.250132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:56.411946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e6da7145534b5b87edb4ea9ca83372'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.447777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e6da7145534b5b87edb4ea9ca83372'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.448840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e6da7145534b5b87edb4ea9ca83372'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.449891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e6da7145534b5b87edb4ea9ca83372'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.469165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-249460eb-7df2-499d-8a48-84b973c3585c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e6da7145534b5b87edb4ea9ca83372'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.491130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a3ccc8b8bc494191ce9629ad4ac242'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.527895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 896/1805] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:55 2022] DELETE /identity/v3/users/a4e6da7145534b5b87edb4ea9ca83372 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:56.530649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b494d-44d6-4af9-b8bd-ad06174fea17 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:56.531209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b494d-44d6-4af9-b8bd-ad06174fea17 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:56.531826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b494d-44d6-4af9-b8bd-ad06174fea17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:56.557189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a3ccc8b8bc494191ce9629ad4ac242'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.577991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7a3b494d-44d6-4af9-b8bd-ad06174fea17 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:56.580145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7a3b494d-44d6-4af9-b8bd-ad06174fea17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:56.600925 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a3ccc8b8bc494191ce9629ad4ac242'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.603528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a3ccc8b8bc494191ce9629ad4ac242'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.605412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a3ccc8b8bc494191ce9629ad4ac242'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:56.643526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:56.708071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 897/1806] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:56.723516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:56.724053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:56.724573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:56.733523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 910/1807] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:26:55 2022] DELETE /identity/v3/users/27a3ccc8b8bc494191ce9629ad4ac242 => generated 0 bytes in 857 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:56.788994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:56.790983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:56.948024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 898/1808] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:56.964495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:57.013119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d9186-500f-4907-91e0-078785dfe717 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:57.039825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:57.040448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.042428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.042428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.042428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:57.042428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:57.044275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:57.092124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 911/1809] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:26:57.099537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:57.128212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21a88ea-403f-409c-8e1f-94bd4d1ee545 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:57.135813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:57.140928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.141375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.141762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.142701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:57.145529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:57.236497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 899/1810] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:26:57 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:57.239864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:57.259940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74bfd4-7da1-4fd2-8eb6-d130f1a1f37e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:57.511764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:57.516836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.518845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.519530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.520995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:57.525482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:57.598602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 912/1811] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:26:57 2022] POST /identity/v3/users => generated 287 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:26:57.604737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:57.632591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc8c73-1739-4209-9a3b-a4ca90ea1ec2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:57.664045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605e513-234b-4c4a-bfd2-4b5377b4f116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:57.665907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e513-234b-4c4a-bfd2-4b5377b4f116 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.666829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e513-234b-4c4a-bfd2-4b5377b4f116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.667821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e513-234b-4c4a-bfd2-4b5377b4f116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.670340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605e513-234b-4c4a-bfd2-4b5377b4f116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:57.697304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605e513-234b-4c4a-bfd2-4b5377b4f116 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:57.747155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 900/1812] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:57.757113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:57.833865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e60c6-d5f3-4249-867b-fb4a2650cf02 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:57.862936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:57.863867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.864251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.864608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.865906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:57.868726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:57.891141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3d3d9-28fc-4ff7-a8a5-7d7fd743cce5 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.891627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3d3d9-28fc-4ff7-a8a5-7d7fd743cce5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.895291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3d3d9-28fc-4ff7-a8a5-7d7fd743cce5 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.895291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 901/1813] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:26:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:26:57.934586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:57.936852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:57.940340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:57.974339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 913/1814] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:26:57.981199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:58.005916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:58.010830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:58.019237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf6945-25bb-4463-a4f4-a8fccb74c0cb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:58.033111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd350b58-5ad4-4fac-b0e9-ec078605fa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:58.037478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350b58-5ad4-4fac-b0e9-ec078605fa53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:58.037809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350b58-5ad4-4fac-b0e9-ec078605fa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:58.038096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd350b58-5ad4-4fac-b0e9-ec078605fa53 admin admin] PATH_INFO: `/v3/projects/3cf302baf3b7484da9af939e59af0abc/users/03f85d8fd69341549899cc1e8d9d5d7d/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:58.102663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd350b58-5ad4-4fac-b0e9-ec078605fa53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf302baf3b7484da9af939e59af0abc, user_id=03f85d8fd69341549899cc1e8d9d5d7d, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:58.109305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd350b58-5ad4-4fac-b0e9-ec078605fa53 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:58.135117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:58.215848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 914/1815] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:26:57 2022] PUT /identity/v3/projects/3cf302baf3b7484da9af939e59af0abc/users/03f85d8fd69341549899cc1e8d9d5d7d/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:58.231914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:58.232801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:58.233349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:58.291284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:58.296315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:58.323821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 902/1816] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:58.434822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 915/1817] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:58.460008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:58.462338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:58.535314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:26:58.598151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785280d4-7428-4289-a4a8-3834747d4ad3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:58.683084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:58.685688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:58.685688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:58.685688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:58.721600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:58.725706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:58.789523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 903/1818] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:58.825624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:58.829134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:59.005206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3f189-a9e3-43f8-8c28-15b009fbb7b7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:59.017537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6359eb9-66f5-42cc-9046-1524925963d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:59.019168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6359eb9-66f5-42cc-9046-1524925963d2 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:59.019862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6359eb9-66f5-42cc-9046-1524925963d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:59.020584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6359eb9-66f5-42cc-9046-1524925963d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:59.052513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6359eb9-66f5-42cc-9046-1524925963d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:59.057600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6359eb9-66f5-42cc-9046-1524925963d2 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:59.125042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 916/1819] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:26:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:26:59.215910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:59.219698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:59.286441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b08e95-35d7-48d2-bf6c-c521cd812f7a service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:59.296151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:59.298261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:59.299132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:59.300031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] PATH_INFO: `/v3/projects/b509253e26f7468d8787fc25ad55dbf4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:59.303566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b509253e26f7468d8787fc25ad55dbf4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:59.308381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:59.447156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b509253e26f7468d8787fc25ad55dbf4', 'request_id': 'req-57752da2-63f8-4803-acfe-6d1152605123'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:59.463323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b509253e26f7468d8787fc25ad55dbf4', 'request_id': 'req-57752da2-63f8-4803-acfe-6d1152605123'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:59.482936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b509253e26f7468d8787fc25ad55dbf4', 'request_id': 'req-57752da2-63f8-4803-acfe-6d1152605123'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:59.536307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] The token cache is being invalidate because project b509253e26f7468d8787fc25ad55dbf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:26:59.538785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:59.540056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-57752da2-63f8-4803-acfe-6d1152605123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:26:59.555965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 904/1820] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:59 2022] DELETE /identity/v3/projects/b509253e26f7468d8787fc25ad55dbf4 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:26:59.557304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:59.557604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:59.558043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:59.638418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:59.642223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:59.808625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 917/1821] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:59.833221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:59.833959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:59.834145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:59.876388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:26:59.880574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:59.903467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 905/1822] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:26:59.905119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:59.906800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:26:59.908447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:26:59.933045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a29a80-0dd4-4ab8-a4eb-659518933033 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:59.959838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:59.960704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:59.960998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:59.961303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:59.962101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:26:59.962417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:59.963982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a61e1-dccf-4d6d-8abf-5bde8dd92790 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:26:59.964381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:26:59.976028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:26:59.978607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:26:59.979085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:26:59.979532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] PATH_INFO: `/v3/projects/6e48ad659b364fb09010d3b81051bb7a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:26:59.981976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e48ad659b364fb09010d3b81051bb7a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:26:59.986604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.004437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 906/1823] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:00.010973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.032802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7668735-b3ca-4a02-89b7-104b5e877123 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.039894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:00.047140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.048356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.050303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.051085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.054740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.061584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.138880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e48ad659b364fb09010d3b81051bb7a', 'request_id': 'req-48ae2cb8-8625-448b-bc18-6272a2a55d9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:00.154243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 907/1824] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:00 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:00.161086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.162464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e48ad659b364fb09010d3b81051bb7a', 'request_id': 'req-48ae2cb8-8625-448b-bc18-6272a2a55d9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:00.189087 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e48ad659b364fb09010d3b81051bb7a', 'request_id': 'req-48ae2cb8-8625-448b-bc18-6272a2a55d9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:00.240170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] The token cache is being invalidate because project 6e48ad659b364fb09010d3b81051bb7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:00.240585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:00.247672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-48ae2cb8-8625-448b-bc18-6272a2a55d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:00.256781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 918/1825] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:26:59 2022] DELETE /identity/v3/projects/6e48ad659b364fb09010d3b81051bb7a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:00.260594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dff14-6f7a-4527-b580-a95179e38a79 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.261085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dff14-6f7a-4527-b580-a95179e38a79 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.261642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dff14-6f7a-4527-b580-a95179e38a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.344735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:00.435124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e46dff14-6f7a-4527-b580-a95179e38a79 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:00.437688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e46dff14-6f7a-4527-b580-a95179e38a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:00.522467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0a028-30c0-4679-a1d6-ad105ebbed08 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.550065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.552844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.553803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.554559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.556191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.564082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.621141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 919/1826] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:00.637282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.638799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 908/1827] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:00 2022] POST /identity/v3/users => generated 291 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:00.642441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.643515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.643681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:00.645333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.669535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503de41e-9794-46ec-af09-e154ba75a02c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.687812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.689089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.689701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.690425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.693055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.698760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:00.699080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.699599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:00.724622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 920/1828] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:00.725708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.725878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 909/1829] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:00.727492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.741619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1452c-843e-44c0-90d4-6407b55561cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.760234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81d0a3-6a63-4659-a48a-99cb8254c473 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.767839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.768868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.769684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.770253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.771835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.774433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.775540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.776725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.777216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.777700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.779135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:00.779724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.799990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.803086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 910/1830] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:00.808110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.842092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb6d6f-c230-4094-bccf-9a4e03ab92fa admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.850913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 921/1831] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:00.854333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e176dec-8a3c-418c-acce-fb0edd2edff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.856264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176dec-8a3c-418c-acce-fb0edd2edff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.856761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176dec-8a3c-418c-acce-fb0edd2edff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.857257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e176dec-8a3c-418c-acce-fb0edd2edff0 admin admin] PATH_INFO: `/v3/projects/44bf19996b7944ca89f78ff129820901/users/b1a4425ab3fe45d8a9ea25cc79436000/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.857426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:00.884216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e176dec-8a3c-418c-acce-fb0edd2edff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bf19996b7944ca89f78ff129820901, user_id=b1a4425ab3fe45d8a9ea25cc79436000, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.887354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e176dec-8a3c-418c-acce-fb0edd2edff0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.903650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87034338-6a94-4c30-90fc-d01c577cc55d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:00.915351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:00.917230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.917836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.918259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:00.919350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:00.921379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:00.953137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 911/1832] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:00 2022] PUT /identity/v3/projects/44bf19996b7944ca89f78ff129820901/users/b1a4425ab3fe45d8a9ea25cc79436000/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:00.956546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:00.957006 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:00.957462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.007269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 922/1833] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:00 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:01.009613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.025750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] MFA Rules not processed for user `b1a4425ab3fe45d8a9ea25cc79436000`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:01.028514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:01.157748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31e26a-04b2-4e54-bdc9-e77ebe31241f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.167418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.168760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.169205 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.169793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.171571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 912/1834] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:01.171792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.175332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.180543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.191201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:01.202182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbb91a-a3b8-455c-8307-fe711a2933f6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.211524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.212403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.212969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.213742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.215378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.216609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 923/1835] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:01 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:01.224050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.225911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.238932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e5e4-0011-402d-8236-a3392d0c6ca6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.245199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.245914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.246458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.246658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.247430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.250247 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.263495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 924/1836] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:27:01.270783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.285556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a69f10-0098-4b81-b48f-9039b01231fd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.290097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 913/1837] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:01 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:01.295307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.402553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:01.437002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb0086-cd33-4730-a2e2-503e7e815933 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.445601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.446809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.447277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.447749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.448990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.452674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.469590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.469861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.470118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.470431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.472134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.474759 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.504034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 925/1838] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:01.506350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.511400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 914/1839] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:01 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:01.512768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.532742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7c41-b136-4546-be14-107a60d72ca5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.537682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d603865f-6366-47a7-a653-d46bb950b075 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.543219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.544616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.545191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.545748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.547514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd3b3b-f9d0-4238-aadd-1931b5b2c270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.547685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.549217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd3b3b-f9d0-4238-aadd-1931b5b2c270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.550724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd3b3b-f9d0-4238-aadd-1931b5b2c270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.551279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd3b3b-f9d0-4238-aadd-1931b5b2c270 admin admin] PATH_INFO: `/v3/projects/bf057b321bd2496d8d556106798916c3/users/f92b72b846ee49d69fdf9ee7f530c9e4/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.551566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.572862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 926/1840] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:01.587089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.599919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd3b3b-f9d0-4238-aadd-1931b5b2c270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf057b321bd2496d8d556106798916c3, user_id=f92b72b846ee49d69fdf9ee7f530c9e4, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.603495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ad489-3e21-4c4e-a347-5d22c3d9fcf2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.604315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd3b3b-f9d0-4238-aadd-1931b5b2c270 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.612513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.615653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.615653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.615653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] PATH_INFO: `/v3/projects/275780bf6a29456ba02e1ad0ea50dbf6/users/44430f6ce19b4843aaad0466bfa4643c/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.632903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 915/1841] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:01 2022] PUT /identity/v3/projects/bf057b321bd2496d8d556106798916c3/users/f92b72b846ee49d69fdf9ee7f530c9e4/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:01.653483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.653908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.654293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.658688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275780bf6a29456ba02e1ad0ea50dbf6, user_id=44430f6ce19b4843aaad0466bfa4643c, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.664801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.686682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 927/1842] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:01 2022] PUT /identity/v3/projects/275780bf6a29456ba02e1ad0ea50dbf6/users/44430f6ce19b4843aaad0466bfa4643c/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:01.691772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.705008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] MFA Rules not processed for user `f92b72b846ee49d69fdf9ee7f530c9e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:01.708011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:01.849660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfb7ce-a1cd-4e7f-90bd-4c397582f188 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.868742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d8f5-fdeb-4a1c-9488-17afba079814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.870479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d8f5-fdeb-4a1c-9488-17afba079814 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.870671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d8f5-fdeb-4a1c-9488-17afba079814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.870969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d8f5-fdeb-4a1c-9488-17afba079814 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.872005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69d8f5-fdeb-4a1c-9488-17afba079814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.881470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69d8f5-fdeb-4a1c-9488-17afba079814 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.897975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 916/1843] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:01.904351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:01.930630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc909d-7860-49b0-8943-55aeca68d00b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:01.931392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 928/1844] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:27:01.933595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.934129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.934986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.942395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:01.944119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:01.944883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:01.945636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:01.947015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:01.952424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:01.991865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] MFA Rules not processed for user `03f85d8fd69341549899cc1e8d9d5d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:01.995427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:02.017094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 917/1845] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:01 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:02.019245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.083598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:02.167470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 929/1846] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:02.169562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.193130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cc27a-14c9-4ca0-a939-a1134ae66d59 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.202311 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ea266-5d68-4acf-8d49-8f0de9c9b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.203974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ea266-5d68-4acf-8d49-8f0de9c9b503 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.204497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae0411-8eb2-4da1-b397-22081a5b42df admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.204993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ea266-5d68-4acf-8d49-8f0de9c9b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.205849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ea266-5d68-4acf-8d49-8f0de9c9b503 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.207515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ea266-5d68-4acf-8d49-8f0de9c9b503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.212173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ea266-5d68-4acf-8d49-8f0de9c9b503 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.214372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.216112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.216679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.217079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] PATH_INFO: `/v3/projects/275780bf6a29456ba02e1ad0ea50dbf6/users/44430f6ce19b4843aaad0466bfa4643c/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.220174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=275780bf6a29456ba02e1ad0ea50dbf6, user_id=44430f6ce19b4843aaad0466bfa4643c, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.224596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.248676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 918/1847] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:02 2022] PUT /identity/v3/projects/275780bf6a29456ba02e1ad0ea50dbf6/users/44430f6ce19b4843aaad0466bfa4643c/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:02.249738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.252039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:02.261883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 930/1848] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:02 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:02.268499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.269319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.269545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.276396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:02.325556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] MFA Rules not processed for user `44430f6ce19b4843aaad0466bfa4643c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:02.330056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:02.372546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d9419-f17d-476a-ab7a-dd7f093c29f2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.381532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.382877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.383442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.384003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.385277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.388956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.428610 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 919/1849] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:02 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:02.429872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.451734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 931/1850] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:02.453093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.467001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499bcda1-eb67-4503-9741-f6813d360720 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.544916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043bcef-52ad-488d-93a2-ed6b9deadb01 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.553021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.554169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.554684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.555135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.556362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.559159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.563224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.563928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.564211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.564487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.566091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.568662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.581224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 920/1851] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:02.581224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.581605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:02.604677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 932/1852] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:02 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:02.604677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.617579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6dcf7-a6d6-41b7-a866-c1a2d196ce59 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.628124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.629228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.629693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.630139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] PATH_INFO: `/v3/projects/a8ebcc2c74ac4d1fb54bfd24136aa52b/users/2621c70483cf417ea9f350241477caf2/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.649870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ebcc2c74ac4d1fb54bfd24136aa52b, user_id=2621c70483cf417ea9f350241477caf2, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.653190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.673892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 933/1853] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:02 2022] PUT /identity/v3/projects/a8ebcc2c74ac4d1fb54bfd24136aa52b/users/2621c70483cf417ea9f350241477caf2/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:02.675456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.681433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43428a-c976-4600-81d4-351d0e5f2a94 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.682851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:02.763201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:02.765048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb2743-521e-4b49-8287-ab618eda105a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.773649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.775241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.775686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.776059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.776236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.777261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.777561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.778099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.778618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.780135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.798329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 934/1854] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:02.798329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.808076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9575d7a-0a3a-4e79-8415-ffb4863480e6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.813473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.814219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.814659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.815114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.816321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.819325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.845900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 935/1855] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:02.847948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.872255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d30a0-5578-41da-b6dc-fdf92c9d681a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.893850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:02.893850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:02.895030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.895216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:02.895216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] PATH_INFO: `/v3/projects/da140836842b40fdb1c4248ef367c0a8/users/8e4f9f6b4473445da694a09483a2c193/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:02.896730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.919552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da140836842b40fdb1c4248ef367c0a8, user_id=8e4f9f6b4473445da694a09483a2c193, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:02.919784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 921/1856] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:02.921098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:02.925758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:02.954285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 936/1857] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:02 2022] PUT /identity/v3/projects/da140836842b40fdb1c4248ef367c0a8/users/8e4f9f6b4473445da694a09483a2c193/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:02.960817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8ba14-914d-48d2-ae3e-a329feb5a84a service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:02.962128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:03.070218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138b578-e880-4721-a6e0-735844ddc062 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:03.078153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:03.079321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:03.079804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:03.080688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:03.082088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:03.085145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:03.109747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 937/1858] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:03.110132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423318e-70f5-44a1-972b-883c1c19740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:03.110132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423318e-70f5-44a1-972b-883c1c19740f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:03.110878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423318e-70f5-44a1-972b-883c1c19740f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:03.111451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423318e-70f5-44a1-972b-883c1c19740f admin admin] PATH_INFO: `/v3/projects/a8ebcc2c74ac4d1fb54bfd24136aa52b/users/2621c70483cf417ea9f350241477caf2/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:03.112527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:03.115899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423318e-70f5-44a1-972b-883c1c19740f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ebcc2c74ac4d1fb54bfd24136aa52b, user_id=2621c70483cf417ea9f350241477caf2, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:03.121857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423318e-70f5-44a1-972b-883c1c19740f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:03.133004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65728639-5597-4693-be55-4b4d84d4ea34 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:03.140689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 922/1859] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:02 2022] PUT /identity/v3/projects/a8ebcc2c74ac4d1fb54bfd24136aa52b/users/2621c70483cf417ea9f350241477caf2/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:03.157089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:03.157878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:03.158351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:03.207709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] MFA Rules not processed for user `2621c70483cf417ea9f350241477caf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:03.209021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:03.283880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:03.283880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:03.283880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:03.283880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] PATH_INFO: `/v3/projects/da140836842b40fdb1c4248ef367c0a8/users/8e4f9f6b4473445da694a09483a2c193/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:03.284329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da140836842b40fdb1c4248ef367c0a8, user_id=8e4f9f6b4473445da694a09483a2c193, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:03.287909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:03.292280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:03.312223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 938/1860] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:03 2022] PUT /identity/v3/projects/da140836842b40fdb1c4248ef367c0a8/users/8e4f9f6b4473445da694a09483a2c193/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:03.355539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 923/1861] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:03.372045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:03.517967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0a7a5-d65a-4a10-9b84-88b32047f10e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:03.525987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:03.526903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:03.527298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:03.527689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:03.687198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:03.692281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:03.731770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 939/1862] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:03.745376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:03.807897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:03.929905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138bbe6-7404-4ff5-9f1b-9fae789c9cee None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:04.132513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44430f6ce19b4843aaad0466bfa4643c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '275780bf6a29456ba02e1ad0ea50dbf6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:04.133924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:04.134383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:04.134709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:04.135479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:04.135787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:04.137626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:04.169719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 924/1863] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:27:04.179356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:04.215581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf81554-0717-4fef-9349-12a8448fd408 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:04.230147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44430f6ce19b4843aaad0466bfa4643c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '275780bf6a29456ba02e1ad0ea50dbf6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:04.233567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:04.234154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:04.234935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:04.235922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:04.238970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:04.332038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:04.346653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 940/1864] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:04 2022] POST /identity/v3/projects => generated 354 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:04.352545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6943db4-b25a-4a75-b398-22d042840527 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:04.462451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44430f6ce19b4843aaad0466bfa4643c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '275780bf6a29456ba02e1ad0ea50dbf6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:04.463519 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:04.464203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:04.464311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:04.465162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:04.465510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:04.467495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:04.492216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 925/1865] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:04.543492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:04.556434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9026ba-1d4a-4608-9997-0c2f1d8b036f tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:04.568246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44430f6ce19b4843aaad0466bfa4643c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '275780bf6a29456ba02e1ad0ea50dbf6', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:04.570397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:04.571459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:04.571939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] PATH_INFO: `/v3/projects/8c99f8eef893455ba44466c9f816dc90` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:04.575506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8c99f8eef893455ba44466c9f816dc90)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:04.579718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:04.581352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:04.693692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c99f8eef893455ba44466c9f816dc90', 'request_id': 'req-e639eb74-172c-4335-8ef4-832e32db88d3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:04.694807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:04.713267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c99f8eef893455ba44466c9f816dc90', 'request_id': 'req-e639eb74-172c-4335-8ef4-832e32db88d3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:04.717384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabc671-12a6-446b-a221-3add80055273 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:04.736380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c99f8eef893455ba44466c9f816dc90', 'request_id': 'req-e639eb74-172c-4335-8ef4-832e32db88d3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:04.767228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] The token cache is being invalidate because project 8c99f8eef893455ba44466c9f816dc90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:04.768093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:04.769738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e639eb74-172c-4335-8ef4-832e32db88d3 tempest-VolumeQuotaClassesTest-671128349 tempest-VolumeQuotaClassesTest-671128349-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:04.774071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 941/1866] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:04 2022] DELETE /identity/v3/projects/8c99f8eef893455ba44466c9f816dc90 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:04.842969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:04.844487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:04.844978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:04.845462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] PATH_INFO: `/v3/users/f92b72b846ee49d69fdf9ee7f530c9e4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:04.848074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92b72b846ee49d69fdf9ee7f530c9e4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:04.858872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:04.992190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0bb3-8c4d-4cf0-92a7-d74cddeecbe2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:04.993783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0bb3-8c4d-4cf0-92a7-d74cddeecbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:04.994458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f0bb3-8c4d-4cf0-92a7-d74cddeecbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:04.999283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92b72b846ee49d69fdf9ee7f530c9e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:05.012747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92b72b846ee49d69fdf9ee7f530c9e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:05.013577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92b72b846ee49d69fdf9ee7f530c9e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:05.023210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92b72b846ee49d69fdf9ee7f530c9e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:05.042277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92b72b846ee49d69fdf9ee7f530c9e4'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:05.043517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:05.100622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 926/1867] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:04 2022] DELETE /identity/v3/users/f92b72b846ee49d69fdf9ee7f530c9e4 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:05.102323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:05.106442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.120344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d05f0bb3-8c4d-4cf0-92a7-d74cddeecbe2 None None] MFA Rules not processed for user `8e4f9f6b4473445da694a09483a2c193`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:05.124781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d05f0bb3-8c4d-4cf0-92a7-d74cddeecbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.325188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f8175-4587-44bd-ba01-8cca23d6b8ab admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:05.340149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 942/1868] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:04 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:05.341382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:05.343303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:05.344112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:05.344357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:05.344679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:05.345088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:05.346813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:05.354507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.393567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:05.394172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.444794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:05.513501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 943/1869] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:05.515572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:05.522448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.531256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:05.535249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:05.554833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 927/1870] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:05 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:05.556580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:05.559887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.560491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8ddbe-d508-4162-bdc0-69e8a2c5ddd6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:05.584768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:05.585924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:05.586347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:05.586822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:05.589694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.807274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:05.812505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:05.875877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d1fd4-1b27-467e-a80c-dd7620b923e5 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:05.878880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 944/1871] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:05.894957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:05.905492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23b675-461d-46bd-887a-db151d43c8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:05.907654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b675-461d-46bd-887a-db151d43c8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:05.908209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:05.917105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b675-461d-46bd-887a-db151d43c8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:05.919205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b675-461d-46bd-887a-db151d43c8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:05.975160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44d835-933c-4b37-a647-dfb2f3207b67 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:05.990124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23b675-461d-46bd-887a-db151d43c8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:05.991992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23b675-461d-46bd-887a-db151d43c8d6 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:05.998975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:05.999302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:05.999840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:06.000427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] PATH_INFO: `/v3/users/b1a4425ab3fe45d8a9ea25cc79436000` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:06.004213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a4425ab3fe45d8a9ea25cc79436000)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:06.008922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:06.035484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 928/1872] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:06.126498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:06.199467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a4425ab3fe45d8a9ea25cc79436000'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:06.218429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a4425ab3fe45d8a9ea25cc79436000'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:06.239760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a4425ab3fe45d8a9ea25cc79436000'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:06.240708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a4425ab3fe45d8a9ea25cc79436000'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:06.241825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a4425ab3fe45d8a9ea25cc79436000'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:06.304750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 945/1873] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:05 2022] DELETE /identity/v3/users/b1a4425ab3fe45d8a9ea25cc79436000 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:06.311714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:06.311941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:06.312072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:06.423081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:06.427040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:06.739398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 929/1874] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:06.798849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:06.816352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:07.021612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6295fcf-e687-4b60-aa80-0c68e34382a8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:07.041160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:07.042218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:07.043008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:07.043668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:07.058654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:07.060487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:07.074963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 946/1875] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:07.780570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:07.784563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:07.812012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:07.817460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:07.819564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7fd29-f5c7-4c76-87c5-356d21a09b36 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:07.837664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:07.841880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:07.842342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:07.842835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] PATH_INFO: `/v3/projects/bf057b321bd2496d8d556106798916c3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:07.857317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf057b321bd2496d8d556106798916c3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:07.860647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:07.871992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da083795-9d7e-4541-a063-2271cafea6ed service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:07.882956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:07.883479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:07.883894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:07.884586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] PATH_INFO: `/v3/projects/44bf19996b7944ca89f78ff129820901` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:07.887060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bf19996b7944ca89f78ff129820901)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:07.892977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:07.970488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf057b321bd2496d8d556106798916c3', 'request_id': 'req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:07.989925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf057b321bd2496d8d556106798916c3', 'request_id': 'req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.005658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bf19996b7944ca89f78ff129820901', 'request_id': 'req-1592176c-524e-4692-b020-0486505ea017'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.008618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf057b321bd2496d8d556106798916c3', 'request_id': 'req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.016801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:08.022297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bf19996b7944ca89f78ff129820901', 'request_id': 'req-1592176c-524e-4692-b020-0486505ea017'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.041144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] The token cache is being invalidate because project bf057b321bd2496d8d556106798916c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:08.041650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.047960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.050342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bf19996b7944ca89f78ff129820901', 'request_id': 'req-1592176c-524e-4692-b020-0486505ea017'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.051461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 930/1876] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:07 2022] DELETE /identity/v3/projects/bf057b321bd2496d8d556106798916c3 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:08.058981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:08.061107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:08.075005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] The token cache is being invalidate because project 44bf19996b7944ca89f78ff129820901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:08.075429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.076350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.081405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 947/1877] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:07 2022] DELETE /identity/v3/projects/44bf19996b7944ca89f78ff129820901 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:08.082820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:08.084949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:08.159244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:08.193425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fcdc0-4f0b-4e1e-9332-cf9baec747e0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:08.281890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592176c-524e-4692-b020-0486505ea017 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:08.288455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:08.289517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:08.289937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:08.290283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] PATH_INFO: `/v3/users/44430f6ce19b4843aaad0466bfa4643c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:08.291806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44430f6ce19b4843aaad0466bfa4643c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:08.294033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:08.414978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:08.418202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:08.419070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:08.421950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] PATH_INFO: `/v3/users/2621c70483cf417ea9f350241477caf2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:08.423027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44430f6ce19b4843aaad0466bfa4643c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.425155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2621c70483cf417ea9f350241477caf2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:08.429807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:08.437290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44430f6ce19b4843aaad0466bfa4643c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.453839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44430f6ce19b4843aaad0466bfa4643c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.455273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44430f6ce19b4843aaad0466bfa4643c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.458591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44430f6ce19b4843aaad0466bfa4643c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.498353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 948/1878] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:08 2022] DELETE /identity/v3/users/44430f6ce19b4843aaad0466bfa4643c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:08.564715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2621c70483cf417ea9f350241477caf2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.584011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2621c70483cf417ea9f350241477caf2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.584011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2621c70483cf417ea9f350241477caf2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.584011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2621c70483cf417ea9f350241477caf2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.593473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2621c70483cf417ea9f350241477caf2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:08.626649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 931/1879] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:08 2022] DELETE /identity/v3/users/2621c70483cf417ea9f350241477caf2 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:09.379694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:09.381815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:09.492037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:09.494396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:09.495364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33696683-1101-4a1d-829f-33d6a3da9884 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:09.541100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:09.609994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:09.635863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:09.636810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:09.637276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:09.637756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] PATH_INFO: `/v3/projects/275780bf6a29456ba02e1ad0ea50dbf6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:09.637933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9eaf5-55c1-4959-b153-f65027186378 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:09.642693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=275780bf6a29456ba02e1ad0ea50dbf6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:09.646052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:09.648877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:09.651327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:09.651740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:09.652351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] PATH_INFO: `/v3/projects/a8ebcc2c74ac4d1fb54bfd24136aa52b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:09.656264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ebcc2c74ac4d1fb54bfd24136aa52b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:09.659027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:09.778684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '275780bf6a29456ba02e1ad0ea50dbf6', 'request_id': 'req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.781132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ebcc2c74ac4d1fb54bfd24136aa52b', 'request_id': 'req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.789687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ebcc2c74ac4d1fb54bfd24136aa52b', 'request_id': 'req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.799792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ebcc2c74ac4d1fb54bfd24136aa52b', 'request_id': 'req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.820881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '275780bf6a29456ba02e1ad0ea50dbf6', 'request_id': 'req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.837005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] The token cache is being invalidate because project a8ebcc2c74ac4d1fb54bfd24136aa52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:09.839785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.842273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6fc0fcf6-e3a1-4ba1-97f0-9dd6181af4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.844809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 932/1880] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:09 2022] DELETE /identity/v3/projects/a8ebcc2c74ac4d1fb54bfd24136aa52b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:09.850184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '275780bf6a29456ba02e1ad0ea50dbf6', 'request_id': 'req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.853276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d748b51-d714-4b73-bcd3-663bb9c3bfc0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:09.854013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d748b51-d714-4b73-bcd3-663bb9c3bfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:09.854653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d748b51-d714-4b73-bcd3-663bb9c3bfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:09.889641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] The token cache is being invalidate because project 275780bf6a29456ba02e1ad0ea50dbf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:09.889641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.890026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-70bafcdb-fa38-4fe1-ae4d-77031869cb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:09.893519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 949/1881] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:09 2022] DELETE /identity/v3/projects/275780bf6a29456ba02e1ad0ea50dbf6 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:09.904062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d0f0e-58b6-4af1-aba9-4a64c8c1d199 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:09.904352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d0f0e-58b6-4af1-aba9-4a64c8c1d199 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:09.904662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d0f0e-58b6-4af1-aba9-4a64c8c1d199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:09.908402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1d748b51-d714-4b73-bcd3-663bb9c3bfc0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:09.913403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1d748b51-d714-4b73-bcd3-663bb9c3bfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:09.948376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dc0d0f0e-58b6-4af1-aba9-4a64c8c1d199 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:09.952167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dc0d0f0e-58b6-4af1-aba9-4a64c8c1d199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:10.021422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 933/1882] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:10.028010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.028705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.031137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.071070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:10.073441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:10.101106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 934/1883] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:10.103676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.125102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 950/1884] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:10.127377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09750b-0e2d-4d16-b3bb-72f380c90dde None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.134909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.135588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.135868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.136082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.136255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.137936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.138129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:10.138129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.138417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.138833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.175079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:10.177054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 935/1885] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:10.184217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:10.184499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.202965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 951/1886] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:10.211150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.212799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804752b9-c43e-4c51-aba4-f7bed8d9160b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.232848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b84e8d3-4035-487d-a93d-10e5d9d27003 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.234296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.235432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.235837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.236429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.237713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.241932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.258985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.260210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.260788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.261361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.262722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:10.263090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.264834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.305653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 952/1887] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:10.313976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.328413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 936/1888] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:10 2022] POST /identity/v3/projects => generated 357 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:10.329793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.339997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187049b-4c3c-43aa-be9f-bd8449bcb570 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.371173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:10.512889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.514581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.514870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.515227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.515958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.518988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.524880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bca90dd-9f8d-41e9-91a1-7f0cd62e59ac admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.535815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.536668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:10.539186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.539933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.540407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.541319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.544648 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.568506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 953/1889] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:10 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:10.576456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.585902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 937/1890] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:10 2022] POST /identity/v3/users => generated 279 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:10.587660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.607923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231f169-2e2d-4335-aac6-a9db0e6cc2ea admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.739143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18bc35a-b04a-4472-ba5b-b2a82164fd8e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.758187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.759257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.759723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.761327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.762459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.765487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.800011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.800011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.800011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.800011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.801482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.804856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.812604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 954/1891] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:10 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:10.821428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.844490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f1b4-4b58-4cb8-9a51-e704cb997f88 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.857749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.860261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.862713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.864713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.867371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.867950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 938/1892] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:10.869505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.872642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.888208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faa971-3c51-4744-9338-9177220c68a3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.909488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 955/1893] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:10.913718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.920299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.921348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.921711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.922108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.923244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.925979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.929800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472819a8-9a7a-4def-8adc-d4b87a86d869 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.938012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.945091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.945091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.945091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:10.945091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:10.945091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:10.960659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 956/1894] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:10.965723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.974618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 939/1895] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:10.979316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22e6cb-1fe9-44b2-b109-64816c70db42 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:10.980261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:10.987092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465eea0-ae17-4098-afff-6b410fa77c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:10.987932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465eea0-ae17-4098-afff-6b410fa77c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:10.988217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465eea0-ae17-4098-afff-6b410fa77c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:10.988489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465eea0-ae17-4098-afff-6b410fa77c5b admin admin] PATH_INFO: `/v3/projects/53906b4936104b94b6d0ea4cb3f855f3/users/5e9b95bd7c504113bee64e514efd16db/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.005823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a587bb-da3b-4f92-9262-a10a6261e9cc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.021346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f436f-713a-4e68-8d5d-9c13bf1c1945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.021617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465eea0-ae17-4098-afff-6b410fa77c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53906b4936104b94b6d0ea4cb3f855f3, user_id=5e9b95bd7c504113bee64e514efd16db, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.023689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f436f-713a-4e68-8d5d-9c13bf1c1945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.024378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f436f-713a-4e68-8d5d-9c13bf1c1945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.025055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f436f-713a-4e68-8d5d-9c13bf1c1945 admin admin] PATH_INFO: `/v3/projects/b874ab0db696418ebcba39fd50bcaf24/users/8ebcaf4085aa408e95072c48500ba213/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.027485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465eea0-ae17-4098-afff-6b410fa77c5b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.051626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 957/1896] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:10 2022] PUT /identity/v3/projects/53906b4936104b94b6d0ea4cb3f855f3/users/5e9b95bd7c504113bee64e514efd16db/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:11.063150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.065724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.065724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.068094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f436f-713a-4e68-8d5d-9c13bf1c1945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b874ab0db696418ebcba39fd50bcaf24, user_id=8ebcaf4085aa408e95072c48500ba213, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.072902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f436f-713a-4e68-8d5d-9c13bf1c1945 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.101737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 940/1897] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:10 2022] PUT /identity/v3/projects/b874ab0db696418ebcba39fd50bcaf24/users/8ebcaf4085aa408e95072c48500ba213/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:11.113877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] MFA Rules not processed for user `5e9b95bd7c504113bee64e514efd16db`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:11.115248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:11.118464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.119023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.119324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.222666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] MFA Rules not processed for user `8ebcaf4085aa408e95072c48500ba213`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:11.226033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:11.260908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 958/1898] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:11.268284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.330622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:11.342309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:11.383015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 941/1899] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:11.390303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.462219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61edbd-93be-4c27-832c-cdbf1604cbe3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.482967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.486664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.488257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.488257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.488793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.493448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.551097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 959/1900] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:11 2022] POST /identity/v3/projects => generated 377 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:11.553627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.575974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27a608-d133-42f3-ae74-6422838a6f90 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.644744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce5715-88cf-4717-9b70-c5b90299c498 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.650924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.651793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.652167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.652535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.653447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.655648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.670741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.671710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.672111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.672502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.674131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.676672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.696420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 960/1901] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:11 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:11.704471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.719819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902d551-c0dd-49ba-ad1e-73cca8f644cd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.721695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 942/1902] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:11 2022] POST /identity/v3/projects => generated 357 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:11.725823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.841588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.842504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.842640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.843017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.843775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.846425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.850852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd5328-90c2-4d55-a5f3-e82f00ae1570 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.855796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.856569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.859224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 961/1903] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:11.864810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.870712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.871995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.873371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.875828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.878217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe9519-983b-4378-9cf5-72d2575b9bd7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.888999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.889947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.890700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.891030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.892358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.894930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.911111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 962/1904] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:11.917174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.917394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 943/1905] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:11 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:11.920103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:11.927337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cf2fe-c288-4dc4-9bb9-574805fd392d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.935402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a8e08-78f5-4097-862a-31ee41083314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.937120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8e08-78f5-4097-862a-31ee41083314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.937491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8e08-78f5-4097-862a-31ee41083314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.937833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8e08-78f5-4097-862a-31ee41083314 admin admin] PATH_INFO: `/v3/projects/ce242acaed5f4f0693d94a4d9f6e1c4b/users/415dd11e489b432bac1225fa03bbbe07/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.940604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f09bfa-5517-4d46-96c3-ccd88a86920d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:11.959403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:11.960459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:11.960904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:11.961236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:11.962338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.966190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.972230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a8e08-78f5-4097-862a-31ee41083314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce242acaed5f4f0693d94a4d9f6e1c4b, user_id=415dd11e489b432bac1225fa03bbbe07, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:11.977526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a8e08-78f5-4097-862a-31ee41083314 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:11.990468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 944/1906] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:11.994218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.006661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:12.014424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:12.026338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 963/1907] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:11 2022] PUT /identity/v3/projects/ce242acaed5f4f0693d94a4d9f6e1c4b/users/415dd11e489b432bac1225fa03bbbe07/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:12.033481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6906d943-7f91-4a6b-b04b-94101111fa62 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.042839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.043316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.044576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.092877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] MFA Rules not processed for user `415dd11e489b432bac1225fa03bbbe07`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:12.094712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:12.131208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.132342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.132804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.133125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] PATH_INFO: `/v3/projects/99154617513f43a7b78a5e861593083d/users/41517dbe39fd4f51a1662cf5fad255e5/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.181175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=99154617513f43a7b78a5e861593083d, user_id=41517dbe39fd4f51a1662cf5fad255e5, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.183977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.208262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 945/1908] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:11 2022] PUT /identity/v3/projects/99154617513f43a7b78a5e861593083d/users/41517dbe39fd4f51a1662cf5fad255e5/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:12.210031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.225239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 964/1909] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:12.231255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.254459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdf818f-354d-4397-9a5b-7732ca97a335 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.327774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b57c9-d50a-4c84-a26d-0449715046ae admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.338170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.338170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.338170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.338170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.338668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.339898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.351255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.351483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.351877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.352251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.353256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.355997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.367747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 946/1910] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:12.374017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.393464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f24c53-a8ad-47d9-a755-5473ef3babd8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.399939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 965/1911] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:12 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:12.403763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.405721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3f501c-4b33-4f8f-a7d8-cd3a6f7d7283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.407483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f501c-4b33-4f8f-a7d8-cd3a6f7d7283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.407900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f501c-4b33-4f8f-a7d8-cd3a6f7d7283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.408211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3f501c-4b33-4f8f-a7d8-cd3a6f7d7283 admin admin] PATH_INFO: `/v3/projects/99154617513f43a7b78a5e861593083d/users/41517dbe39fd4f51a1662cf5fad255e5/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.413634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f501c-4b33-4f8f-a7d8-cd3a6f7d7283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99154617513f43a7b78a5e861593083d, user_id=41517dbe39fd4f51a1662cf5fad255e5, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.417894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3f501c-4b33-4f8f-a7d8-cd3a6f7d7283 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.453259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 947/1912] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:12 2022] PUT /identity/v3/projects/99154617513f43a7b78a5e861593083d/users/41517dbe39fd4f51a1662cf5fad255e5/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:12.455111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.461883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.461883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.518573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] MFA Rules not processed for user `41517dbe39fd4f51a1662cf5fad255e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:12.519917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:12.530757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93eee67-e596-432e-bfef-aa7a0f5dacd1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.559268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:12.610452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:12.643271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 948/1913] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:12.654215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.656529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:12.663119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.663829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.664118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.664408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.665118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.667439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.695089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 966/1914] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:12 2022] POST /identity/v3/users => generated 296 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:12.699881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.715528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe3d65-a952-49d6-a5ce-12aa392339b1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.723831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.724536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.724981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.725706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.726809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.728636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.745971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 967/1915] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:27:12.746380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8529d-902f-4622-9924-d1036c2ee214 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.754194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.760539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.761412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.761899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.762305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.769476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149cd6-b050-4709-8c54-f21adb9905b7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.774949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:12.775806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.776288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:12.776578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:12.776861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] PATH_INFO: `/v3/projects/604da2814721418baaba58bb508389a3/users/e4097d5425984d62a3e562c668f8811b/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:12.779865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.807099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604da2814721418baaba58bb508389a3, user_id=e4097d5425984d62a3e562c668f8811b, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:12.808169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 949/1916] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:12.812033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:12.838394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 968/1917] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:12 2022] PUT /identity/v3/projects/604da2814721418baaba58bb508389a3/users/e4097d5425984d62a3e562c668f8811b/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:12.847829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:12.866059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c2e67-35c7-4865-9f41-ecd1ea667bf9 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:12.975627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:13.000757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:13.002558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.003544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.004926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.005542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:13.009162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:13.040183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 950/1918] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:27:13.043157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:13.078043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d873f87c-eac5-473c-813b-f213bf10a9dc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:13.088393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580aa64-1e6c-4783-a3cd-dc73cadff039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:13.090084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580aa64-1e6c-4783-a3cd-dc73cadff039 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.090717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580aa64-1e6c-4783-a3cd-dc73cadff039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.091960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580aa64-1e6c-4783-a3cd-dc73cadff039 admin admin] PATH_INFO: `/v3/projects/604da2814721418baaba58bb508389a3/users/e4097d5425984d62a3e562c668f8811b/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.096892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580aa64-1e6c-4783-a3cd-dc73cadff039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604da2814721418baaba58bb508389a3, user_id=e4097d5425984d62a3e562c668f8811b, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:13.102953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c580aa64-1e6c-4783-a3cd-dc73cadff039 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:13.122041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce92df-9466-4791-9e48-6f49efe79fe9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:13.132355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 951/1919] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:13 2022] PUT /identity/v3/projects/604da2814721418baaba58bb508389a3/users/e4097d5425984d62a3e562c668f8811b/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:13.147662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.148564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.149076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.217350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] MFA Rules not processed for user `e4097d5425984d62a3e562c668f8811b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:13.222578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:13.280679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b91501-cab3-436a-9630-3d94d9b04863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:13.281734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91501-cab3-436a-9630-3d94d9b04863 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.282195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91501-cab3-436a-9630-3d94d9b04863 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.285495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b91501-cab3-436a-9630-3d94d9b04863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.365778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 952/1920] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:13.417464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:13.456675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b91501-cab3-436a-9630-3d94d9b04863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:13.461572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b91501-cab3-436a-9630-3d94d9b04863 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:13.465025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:13.467658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:13.487215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:13.495738 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 969/1921] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:13.500297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa4a1c-64bc-4d92-a054-a3f7f80d17c6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:13.511778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:13.518868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.519273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.519864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.525276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e63d1-9497-4e8b-9701-5c521b5674da None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.525820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e63d1-9497-4e8b-9701-5c521b5674da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.526344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e63d1-9497-4e8b-9701-5c521b5674da None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.529337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 970/1922] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:27:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 15:27:13.545900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.546351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.546828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.563707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:13.585623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:13.607306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:13.611328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:13.623699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 953/1923] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:13.778457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:13.802473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571b12a-f96f-4a4f-88cd-48c573390657 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:13.811980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:13.813323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.813698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.814228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.834507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 971/1924] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:13.863682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:13.866448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:13.900263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac4156-1279-4a0b-94f8-e7e778786c36 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:13.912981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:13.914290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:13.914864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:13.915362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:13.946897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:13.951900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:13.960637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:13.962485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:13.978617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 954/1925] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:13.980873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 972/1926] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:14.015123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:14.036772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b661fa-1f50-4a20-a7c3-3777b6f65873 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:14.045502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:14.046658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:14.047506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:14.047849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:14.052920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:14.178616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:14.181419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7162612d-818f-4aa3-9326-76360f6684b2 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:14.198504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 955/1927] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:22.539611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:22.671953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa6e09-2181-43e8-a0ad-fe83410b7ce4 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:22.680544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:22.682128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:22.682662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:22.683189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] PATH_INFO: `/v3/users/5e9b95bd7c504113bee64e514efd16db` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:22.685080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e9b95bd7c504113bee64e514efd16db)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:22.687162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:22.792360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e9b95bd7c504113bee64e514efd16db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:22.811820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e9b95bd7c504113bee64e514efd16db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:22.828854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e9b95bd7c504113bee64e514efd16db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:22.831354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e9b95bd7c504113bee64e514efd16db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:22.833243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e9b95bd7c504113bee64e514efd16db'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:22.837358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:22.878210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 973/1928] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:22 2022] DELETE /identity/v3/users/5e9b95bd7c504113bee64e514efd16db => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:22.884921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:22.885812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:22.885974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:22.914883 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:22.916787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:23.028114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 956/1929] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:23.042953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:23.044955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:23.147552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57411d03-8f01-4639-81a0-1a51946fb718 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:23.154143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:23.155393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:23.155919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:23.156410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:23.178871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:23.181773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:23.201173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 974/1930] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:23.623847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:23.627618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:23.657401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad67f1-0fd0-4c30-9f87-04aa88353dd9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:23.664010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:23.665090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:23.665575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:23.665753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] PATH_INFO: `/v3/projects/53906b4936104b94b6d0ea4cb3f855f3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:23.667963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53906b4936104b94b6d0ea4cb3f855f3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:23.671142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:23.706384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:23.747759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53906b4936104b94b6d0ea4cb3f855f3', 'request_id': 'req-5dd1f489-191a-4965-9001-ea4db2048068'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:23.755774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53906b4936104b94b6d0ea4cb3f855f3', 'request_id': 'req-5dd1f489-191a-4965-9001-ea4db2048068'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:23.762210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53906b4936104b94b6d0ea4cb3f855f3', 'request_id': 'req-5dd1f489-191a-4965-9001-ea4db2048068'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:23.782447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] The token cache is being invalidate because project 53906b4936104b94b6d0ea4cb3f855f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:23.783031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:23.784148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:23.791379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 957/1931] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:23 2022] DELETE /identity/v3/projects/53906b4936104b94b6d0ea4cb3f855f3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:23.792823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:23.812275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ffd2b-22d6-4587-823b-9d7c727a35ea service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:23.918235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:23.919283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:23.919638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:23.919991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] PATH_INFO: `/v3/users/415dd11e489b432bac1225fa03bbbe07` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:23.921331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415dd11e489b432bac1225fa03bbbe07)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:23.923760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:24.004560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415dd11e489b432bac1225fa03bbbe07'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.018929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415dd11e489b432bac1225fa03bbbe07'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.032873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415dd11e489b432bac1225fa03bbbe07'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.033929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415dd11e489b432bac1225fa03bbbe07'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.035233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415dd11e489b432bac1225fa03bbbe07'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.071957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 975/1932] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:23 2022] DELETE /identity/v3/users/415dd11e489b432bac1225fa03bbbe07 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:24.633331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:24.636602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:24.665265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f489-191a-4965-9001-ea4db2048068 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:24.759032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:24.760161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:24.760454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:24.760861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] PATH_INFO: `/v3/projects/ce242acaed5f4f0693d94a4d9f6e1c4b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:24.762553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce242acaed5f4f0693d94a4d9f6e1c4b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:24.764632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:24.829614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce242acaed5f4f0693d94a4d9f6e1c4b', 'request_id': 'req-b43b60db-588d-4aa6-adb2-b431cbeb7d08'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.837621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce242acaed5f4f0693d94a4d9f6e1c4b', 'request_id': 'req-b43b60db-588d-4aa6-adb2-b431cbeb7d08'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.848406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce242acaed5f4f0693d94a4d9f6e1c4b', 'request_id': 'req-b43b60db-588d-4aa6-adb2-b431cbeb7d08'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.869509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] The token cache is being invalidate because project ce242acaed5f4f0693d94a4d9f6e1c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:24.870016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.870830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:24.872600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 958/1933] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:24 2022] DELETE /identity/v3/projects/ce242acaed5f4f0693d94a4d9f6e1c4b => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:24.877105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:24.878844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:24.892280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:24.897946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9758f82-b5db-47c4-a95d-d7cae1cdab59 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:24.989175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:24.990946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:24.991438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:24.992014 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] PATH_INFO: `/v3/users/e4097d5425984d62a3e562c668f8811b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:24.995032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4097d5425984d62a3e562c668f8811b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:24.998159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:25.079886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4097d5425984d62a3e562c668f8811b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.087859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4097d5425984d62a3e562c668f8811b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.104061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4097d5425984d62a3e562c668f8811b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.104934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4097d5425984d62a3e562c668f8811b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.105784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-23a4fad0-56ac-43ad-95f5-18fc0503064d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4097d5425984d62a3e562c668f8811b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.134428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 976/1934] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:24 2022] DELETE /identity/v3/users/e4097d5425984d62a3e562c668f8811b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:25.762201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:25.764123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:25.781845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b60db-588d-4aa6-adb2-b431cbeb7d08 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:25.834414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:25.871865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:25.872865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:25.873160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:25.873437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] PATH_INFO: `/v3/projects/604da2814721418baaba58bb508389a3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:25.874901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604da2814721418baaba58bb508389a3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:25.877245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:25.954497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604da2814721418baaba58bb508389a3', 'request_id': 'req-80da53da-68d7-45e6-a506-625a5afd0985'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.964453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604da2814721418baaba58bb508389a3', 'request_id': 'req-80da53da-68d7-45e6-a506-625a5afd0985'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.974643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604da2814721418baaba58bb508389a3', 'request_id': 'req-80da53da-68d7-45e6-a506-625a5afd0985'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.996206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] The token cache is being invalidate because project 604da2814721418baaba58bb508389a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:25.996649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.997969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-80da53da-68d7-45e6-a506-625a5afd0985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:25.999917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 959/1935] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:25 2022] DELETE /identity/v3/projects/604da2814721418baaba58bb508389a3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:26.008641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.009529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.009529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.056336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:26.058809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:26.195375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 977/1936] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:26.203589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.204106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.204602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.239399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:26.243403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:26.269741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 960/1937] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:26.277386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:26.296355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc4696-b414-4dc7-8ecd-f8e4226f3d54 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:26.305208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:26.305966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.306355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.306716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.308090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:26.308650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:26.312148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:26.333412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 978/1938] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:26.342474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:26.355827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea555e-c065-4682-ad85-3327d5619913 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:26.362985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:26.363942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.364885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.364885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.365950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:26.369057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:26.425490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 961/1939] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:26 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:26.432565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:26.446933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86cb73-ba8a-4e3f-ad4a-0ded6420c8e1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:26.460583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:26.544726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:26.545438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.545727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.546024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.546903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:26.549449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:26.598316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:26.602832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 979/1940] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:26 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:26.632634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd9ed6-6fae-4817-aef6-a76c16f6ab87 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:26.654831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:26.656061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.656527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.657360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.658137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:26.660204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:26.680130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 962/1941] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:26.685219 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:26.710686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1656cde-0d25-4451-844c-06232108c8e2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:26.719750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7123ced-ae8b-468e-b323-d05e397d669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:26.720986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123ced-ae8b-468e-b323-d05e397d669a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.721498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123ced-ae8b-468e-b323-d05e397d669a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.722017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7123ced-ae8b-468e-b323-d05e397d669a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.723488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7123ced-ae8b-468e-b323-d05e397d669a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:26.726725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7123ced-ae8b-468e-b323-d05e397d669a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:26.741048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 980/1942] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:26.746921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:26.768026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d6cb4-6be6-4ec6-ba18-861bd80ab562 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:26.777835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:26.779644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.780273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.780875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] PATH_INFO: `/v3/projects/b9f57d4cee2f44758b1f3294b232ed76/users/59032551937b4b2b8ffd9f40031c175b/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:26.840925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f57d4cee2f44758b1f3294b232ed76, user_id=59032551937b4b2b8ffd9f40031c175b, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:26.848467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:26.900744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 963/1943] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:26 2022] PUT /identity/v3/projects/b9f57d4cee2f44758b1f3294b232ed76/users/59032551937b4b2b8ffd9f40031c175b/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:26.926415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421579-0758-450e-8a30-e7787b623f22 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:26.927082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421579-0758-450e-8a30-e7787b623f22 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:26.927401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421579-0758-450e-8a30-e7787b623f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:27.007766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-46421579-0758-450e-8a30-e7787b623f22 None None] MFA Rules not processed for user `59032551937b4b2b8ffd9f40031c175b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:27.009544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-46421579-0758-450e-8a30-e7787b623f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:27.130821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 981/1944] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:27.145829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:27.149369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:27.356578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789b253-f9ed-467d-9c69-c5ead237aff6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:27.378361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f107f3-b047-437e-a25a-1629626d6526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:27.379682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f107f3-b047-437e-a25a-1629626d6526 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:27.379981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f107f3-b047-437e-a25a-1629626d6526 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:27.380321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f107f3-b047-437e-a25a-1629626d6526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:27.397532 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f107f3-b047-437e-a25a-1629626d6526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:27.400549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f107f3-b047-437e-a25a-1629626d6526 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:27.418018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 964/1945] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:37.119115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:37.409935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:42.661843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421579-0758-450e-8a30-e7787b623f22 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:42.869731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421579-0758-450e-8a30-e7787b623f22 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:42.887321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:42.887321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:42.887321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:42.888500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] PATH_INFO: `/v3/users/59032551937b4b2b8ffd9f40031c175b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:42.890901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=59032551937b4b2b8ffd9f40031c175b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:42.896008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:43.065972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59032551937b4b2b8ffd9f40031c175b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:43.083021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59032551937b4b2b8ffd9f40031c175b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:43.097120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59032551937b4b2b8ffd9f40031c175b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:43.100678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59032551937b4b2b8ffd9f40031c175b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:43.102084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59032551937b4b2b8ffd9f40031c175b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:43.141695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 982/1946] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:42 2022] DELETE /identity/v3/users/59032551937b4b2b8ffd9f40031c175b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:43.158579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:43.159318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:43.159819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:43.184396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:43.186461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:43.333340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 965/1947] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:43.350837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:43.353903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:43.574877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c50de-17d0-4527-bc4e-165e53a506aa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:43.599912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a55d-be35-4e60-a58f-dc7b80e0d6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:43.604769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a55d-be35-4e60-a58f-dc7b80e0d6de service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:43.604769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a55d-be35-4e60-a58f-dc7b80e0d6de service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:43.604769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a55d-be35-4e60-a58f-dc7b80e0d6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:43.662348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a55d-be35-4e60-a58f-dc7b80e0d6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:43.671034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a55d-be35-4e60-a58f-dc7b80e0d6de service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:43.724430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 983/1948] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:44.436232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:44.439835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:44.475039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533d614-a556-4b0f-baee-9c52a03bfe45 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:44.499092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:44.500334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:44.500834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:44.501556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] PATH_INFO: `/v3/projects/b9f57d4cee2f44758b1f3294b232ed76` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:44.509829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f57d4cee2f44758b1f3294b232ed76)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:44.519911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:44.663698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f57d4cee2f44758b1f3294b232ed76', 'request_id': 'req-a9056e8d-5d25-428b-b0eb-b4bce754df45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:44.674372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f57d4cee2f44758b1f3294b232ed76', 'request_id': 'req-a9056e8d-5d25-428b-b0eb-b4bce754df45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:44.688861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f57d4cee2f44758b1f3294b232ed76', 'request_id': 'req-a9056e8d-5d25-428b-b0eb-b4bce754df45'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:44.718233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] The token cache is being invalidate because project b9f57d4cee2f44758b1f3294b232ed76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:44.718233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:44.718233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a9056e8d-5d25-428b-b0eb-b4bce754df45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:44.718233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 966/1949] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:44 2022] DELETE /identity/v3/projects/b9f57d4cee2f44758b1f3294b232ed76 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:44.725389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:44.725916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:44.726442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:44.754862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:44.756306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:44.785539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:44.867333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 984/1950] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:44.876758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:44.877325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:44.877749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:44.911737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:44.914690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:44.932616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 967/1951] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:44.946164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:44.967150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cb1f6-cc04-4286-ad76-386f60a07379 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:44.975333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:44.976139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:44.976541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:44.976930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:44.978203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:44.978715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:44.980579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:44.998788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 985/1952] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:45.001485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.019807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0f6c9-e4d9-42ab-97a8-a8ee7f67eebc None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:45.025188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:45.025879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.026167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.026436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.027271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:45.029211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:45.071918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 968/1953] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:45 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:45.081690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.093217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c1bf-3542-43e9-a502-36ed8fe7fae9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:45.179040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:45.179764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.180249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.180529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.181277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:45.183443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:45.216363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 986/1954] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:45 2022] POST /identity/v3/users => generated 286 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:45.222456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.237197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3f363-1954-4e89-b7ef-e1273e0cc688 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:45.242860 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:45.244068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.244068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.244068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.244836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:45.246567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:45.261462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 969/1955] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:45.266879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.283133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcadab-6a70-41b9-8f6b-b341ef1e4c50 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:45.294554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7541f4c-ab8f-42b0-a18f-4d1201ceff4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:45.296051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7541f4c-ab8f-42b0-a18f-4d1201ceff4b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.296645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7541f4c-ab8f-42b0-a18f-4d1201ceff4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.297329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7541f4c-ab8f-42b0-a18f-4d1201ceff4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.299244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7541f4c-ab8f-42b0-a18f-4d1201ceff4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:45.301573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7541f4c-ab8f-42b0-a18f-4d1201ceff4b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:45.318262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 987/1956] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:45.324387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.343156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad8762-9f05-457f-a348-d866229d1638 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:45.351175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:45.357064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.357064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.357064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] PATH_INFO: `/v3/projects/e9a487ffeb8544f7b17c08b2d2003d23/users/bd0b6099e1a54cd68d9841a76d599d0e/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.393711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a487ffeb8544f7b17c08b2d2003d23, user_id=bd0b6099e1a54cd68d9841a76d599d0e, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:45.398738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:45.440797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 970/1957] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:45 2022] PUT /identity/v3/projects/e9a487ffeb8544f7b17c08b2d2003d23/users/bd0b6099e1a54cd68d9841a76d599d0e/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:45.453634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.453919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.454197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.502199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] MFA Rules not processed for user `bd0b6099e1a54cd68d9841a76d599d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:45.503622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:45.608407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 988/1958] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:45.622716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.623599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:45.664159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:45.725025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bcd0a-e5e3-4d66-b6f2-d000fbef061a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:45.746498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e3da8-059a-4a05-897f-8cd3d1922e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:45.748288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3da8-059a-4a05-897f-8cd3d1922e72 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:45.749412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3da8-059a-4a05-897f-8cd3d1922e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:45.750366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e3da8-059a-4a05-897f-8cd3d1922e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:45.781356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3da8-059a-4a05-897f-8cd3d1922e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:45.784616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e3da8-059a-4a05-897f-8cd3d1922e72 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:45.805996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 971/1959] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:45.850171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:45.908196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:46.043707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c01a27-fe38-4540-877e-296e09f51dd8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:46.067880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:46.069343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:46.069927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:46.070426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] PATH_INFO: `/v3/users/bd0b6099e1a54cd68d9841a76d599d0e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:46.079504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0b6099e1a54cd68d9841a76d599d0e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:46.080774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:46.187324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0b6099e1a54cd68d9841a76d599d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.200540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0b6099e1a54cd68d9841a76d599d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.223191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0b6099e1a54cd68d9841a76d599d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.226096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0b6099e1a54cd68d9841a76d599d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.227321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0b6099e1a54cd68d9841a76d599d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.270181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 989/1960] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:45 2022] DELETE /identity/v3/users/bd0b6099e1a54cd68d9841a76d599d0e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:46.277465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:46.277961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:46.278446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:46.325766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:46.329378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:46.470790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 972/1961] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:46.481941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:46.483492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:46.590375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:46.591940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:46.612896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7706c788-9d3f-43f9-9cff-e34939a8107b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:46.623889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:46.626759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:46.627406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f2e98-0a7e-4741-8643-b24f836b1086 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:46.628661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:46.629403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] PATH_INFO: `/v3/users/51d737907e404f28a6770f588fe7c651` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:46.632698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d737907e404f28a6770f588fe7c651)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:46.638623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:46.643155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ee0c4-3227-4340-8256-10ae9e95e7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:46.646199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee0c4-3227-4340-8256-10ae9e95e7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:46.651316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee0c4-3227-4340-8256-10ae9e95e7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:46.651819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ee0c4-3227-4340-8256-10ae9e95e7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:46.666745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ee0c4-3227-4340-8256-10ae9e95e7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:46.670704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ee0c4-3227-4340-8256-10ae9e95e7d6 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:46.695173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 990/1962] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:46.743387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d737907e404f28a6770f588fe7c651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.758242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d737907e404f28a6770f588fe7c651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.763669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d737907e404f28a6770f588fe7c651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.765045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d737907e404f28a6770f588fe7c651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.781872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d737907e404f28a6770f588fe7c651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:46.842720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 973/1963] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:46 2022] DELETE /identity/v3/users/51d737907e404f28a6770f588fe7c651 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:46.844813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:46.845291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:46.845723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:46.866493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:46.868131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:46.934877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:46.990782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 991/1964] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:47.004801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:47.006426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:47.110636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:47.135874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:47.140764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:47.181706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109484b5-e91b-49cd-82ef-f4e608ce1d30 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:47.183479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46608f48-7f57-4eaa-bdab-d324afd53691 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:47.200571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:47.201864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:47.202881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:47.203506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:47.203795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:47.204112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:47.204413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:47.205330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] PATH_INFO: `/v3/projects/e9a487ffeb8544f7b17c08b2d2003d23` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:47.209125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a487ffeb8544f7b17c08b2d2003d23)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:47.217634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:47.246024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:47.253800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:47.292124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 974/1965] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:47.381388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a487ffeb8544f7b17c08b2d2003d23', 'request_id': 'req-8adf804c-7d5f-4b82-9037-51cb438ef864'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:47.399186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a487ffeb8544f7b17c08b2d2003d23', 'request_id': 'req-8adf804c-7d5f-4b82-9037-51cb438ef864'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:47.416403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a487ffeb8544f7b17c08b2d2003d23', 'request_id': 'req-8adf804c-7d5f-4b82-9037-51cb438ef864'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:47.448174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] The token cache is being invalidate because project e9a487ffeb8544f7b17c08b2d2003d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:47.449189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:47.451154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:47.478693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 992/1966] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:47 2022] DELETE /identity/v3/projects/e9a487ffeb8544f7b17c08b2d2003d23 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:48.205711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:48.208161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:48.264304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9de03-278b-426b-b4f4-400d887c48a0 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:48.429611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:48.431819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:48.432313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:48.433510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] PATH_INFO: `/v3/projects/e3b59a0d413f4edd8e465e2f365fa6f0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:48.442397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b59a0d413f4edd8e465e2f365fa6f0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:48.445902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:48.603038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b59a0d413f4edd8e465e2f365fa6f0', 'request_id': 'req-b87348a3-f630-4031-a140-8700183134aa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:48.630788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b59a0d413f4edd8e465e2f365fa6f0', 'request_id': 'req-b87348a3-f630-4031-a140-8700183134aa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:48.637608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b59a0d413f4edd8e465e2f365fa6f0', 'request_id': 'req-b87348a3-f630-4031-a140-8700183134aa'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:48.660473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] The token cache is being invalidate because project e3b59a0d413f4edd8e465e2f365fa6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:48.661102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:48.662224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:48.666826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 975/1967] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:48 2022] DELETE /identity/v3/projects/e3b59a0d413f4edd8e465e2f365fa6f0 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:48.671214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:48.673174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:48.699678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf804c-7d5f-4b82-9037-51cb438ef864 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:48.881587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:48.883071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:48.883438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:48.883971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] PATH_INFO: `/v3/users/1c62c41ab7ff4072b4132cf66f62b22a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:48.887107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c62c41ab7ff4072b4132cf66f62b22a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:48.892080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:49.036492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c62c41ab7ff4072b4132cf66f62b22a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:49.055115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c62c41ab7ff4072b4132cf66f62b22a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:49.070174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c62c41ab7ff4072b4132cf66f62b22a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:49.071068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c62c41ab7ff4072b4132cf66f62b22a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:49.073262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c62c41ab7ff4072b4132cf66f62b22a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:49.083275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:49.113543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 993/1968] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:48 2022] DELETE /identity/v3/users/1c62c41ab7ff4072b4132cf66f62b22a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:50.525362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:50.532875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:50.563936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87348a3-f630-4031-a140-8700183134aa admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:50.661863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:50.667166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:50.718667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:50.720932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:50.720932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:50.721253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] PATH_INFO: `/v3/projects/c3f4526f78514a67a42403a281f384da` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:50.723728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f4526f78514a67a42403a281f384da)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:50.726653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:50.757171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:50.812583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d83511-4f75-4e3b-8d21-7050eb41f7c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:50.838681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f4526f78514a67a42403a281f384da', 'request_id': 'req-b2dfee55-0318-465c-a172-fe27955b0a9a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:50.854917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f4526f78514a67a42403a281f384da', 'request_id': 'req-b2dfee55-0318-465c-a172-fe27955b0a9a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:50.867393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f4526f78514a67a42403a281f384da', 'request_id': 'req-b2dfee55-0318-465c-a172-fe27955b0a9a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:50.886073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] The token cache is being invalidate because project c3f4526f78514a67a42403a281f384da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:50.886369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:50.887531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b2dfee55-0318-465c-a172-fe27955b0a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:50.890012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 976/1969] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:50 2022] DELETE /identity/v3/projects/c3f4526f78514a67a42403a281f384da => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:50.898374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95509c4a-cdb7-4738-bfdc-3dde021e9fb7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:50.899049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95509c4a-cdb7-4738-bfdc-3dde021e9fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:50.899487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95509c4a-cdb7-4738-bfdc-3dde021e9fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:50.946954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-95509c4a-cdb7-4738-bfdc-3dde021e9fb7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:50.948417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-95509c4a-cdb7-4738-bfdc-3dde021e9fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:50.953972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:50.955567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:50.956063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:50.956557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] PATH_INFO: `/v3/users/8ebcaf4085aa408e95072c48500ba213` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:50.960068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ebcaf4085aa408e95072c48500ba213)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:50.964071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:50.985066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 977/1970] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:50.995848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:50.997574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:50.997753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:51.082435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:51.087900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:51.132152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ebcaf4085aa408e95072c48500ba213'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:51.144413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 978/1971] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:51.149903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:51.171399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ebcaf4085aa408e95072c48500ba213'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:51.174249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704beb9-5544-4328-b43c-ff81f48d9596 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:51.185372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:51.186893 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:51.187175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:51.187599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:51.193446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:51.194711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:51.196091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ebcaf4085aa408e95072c48500ba213'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:51.197128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ebcaf4085aa408e95072c48500ba213'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:51.198175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa37189c-40f3-4e33-ad95-4c18339be456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ebcaf4085aa408e95072c48500ba213'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:51.199955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:51.260071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 979/1972] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:51.267331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:51.276265 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:51.287236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 994/1973] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:50 2022] DELETE /identity/v3/users/8ebcaf4085aa408e95072c48500ba213 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:51.303018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:51.305301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:51.307322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:51.348901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f231d-8639-4e04-bb05-e7ee73e3ab82 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:51.384045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:51.385752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:51.407128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:51.610830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:51.612005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:51.612510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:51.612993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:51.614734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:51.624430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:51.660860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 995/1974] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:51.694951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:51.702391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:51.771534 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 980/1975] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:51 2022] POST /identity/v3/projects => generated 409 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:51.773804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:51.903690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00973a-5901-4ce6-886d-3f640ef359f7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:52.103229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:52.104371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:52.104874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:52.105352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:52.112916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:52.139256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305a090-3953-4a11-8e8c-ef282235e4f4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:52.143334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:52.149858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:52.171561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 996/1976] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:52.207882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:52.215039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:52.215552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:52.216302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:52.231529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:52.235449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:52.346717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 981/1977] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:51 2022] POST /identity/v3/users => generated 305 bytes in 574 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:52.353346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:52.379551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27b22c-e0eb-44dd-a570-87f2817ec11e service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:52.398390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:52.399128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:52.399448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:52.399868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:52.406632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:52.408446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:52.440334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 997/1978] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:52.445569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:52.472354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d7d7d-58f4-41ee-b66f-cbdb13271f7b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:52.489420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f793b2-7dad-4772-80bf-5854c0e97cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:52.490173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f793b2-7dad-4772-80bf-5854c0e97cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:52.490469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f793b2-7dad-4772-80bf-5854c0e97cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:52.490820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f793b2-7dad-4772-80bf-5854c0e97cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:52.491626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f793b2-7dad-4772-80bf-5854c0e97cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:52.493342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f793b2-7dad-4772-80bf-5854c0e97cfa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:52.528609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 982/1979] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:52.533447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:52.556287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c474bc1-ba0c-4fac-967c-9297f46b8b50 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:52.566149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:52.568120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:52.568637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:52.568931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] PATH_INFO: `/v3/projects/af907554b4264e3e8fc4c9014fff066f/users/790ca005d18a4424841f7fb88a7550bd/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:52.610909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=af907554b4264e3e8fc4c9014fff066f, user_id=790ca005d18a4424841f7fb88a7550bd, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:52.622997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:52.672145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 998/1980] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:52 2022] PUT /identity/v3/projects/af907554b4264e3e8fc4c9014fff066f/users/790ca005d18a4424841f7fb88a7550bd/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:52.692376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:52.693710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:52.694201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:52.775039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] MFA Rules not processed for user `790ca005d18a4424841f7fb88a7550bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:52.777737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:53.059571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:53.060992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 983/1981] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:53.234432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:53.400173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0a6a2-9640-4316-8fc0-2e9c3a203e0b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:53.418609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:53.421204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:53.423928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:53.424348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:53.425410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:53.432351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:53.527722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 999/1982] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:53 2022] POST /identity/v3/projects => generated 409 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:53.537686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:53.577554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c34d6-4983-4c6e-a0bd-8ccaee68359c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:53.743984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:53.747173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:53.747904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:53.748614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:53.751668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:53.756109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:53.820332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 984/1983] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:53 2022] POST /identity/v3/users => generated 311 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:53.821892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:53.850653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a803cf4d-8b2a-4423-b3c7-dca75194ae07 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:53.868999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:53.870750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:53.870750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:53.870750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:53.871455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:53.874279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:53.928827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1000/1984] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:53.931095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:53.955281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2b4d1-3dfb-40dc-8f9a-1947d6527960 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:53.980675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:53.982753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:53.983466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:53.984075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] PATH_INFO: `/v3/projects/6518b545f2ff43389a1aaee061d468ee/users/eee37b2b8a714aeda9cce8488d5672af/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:54.076348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6518b545f2ff43389a1aaee061d468ee, user_id=eee37b2b8a714aeda9cce8488d5672af, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:54.084114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:54.108920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:54.113720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:54.137533 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 985/1985] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:53 2022] PUT /identity/v3/projects/6518b545f2ff43389a1aaee061d468ee/users/eee37b2b8a714aeda9cce8488d5672af/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:54.139676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:54.207778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d9d80-23cb-4794-8fd4-23276408d568 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:54.259939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:54.374280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0dba9-14ff-47cb-b0da-67ef40882fe0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:54.382355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:54.384064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:54.384896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:54.385589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] PATH_INFO: `/v3/projects/b874ab0db696418ebcba39fd50bcaf24` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:54.387611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:54.388700 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b874ab0db696418ebcba39fd50bcaf24)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:54.389596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:54.389974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:54.391203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:54.392291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:54.393364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:54.398785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:54.422871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 986/1986] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:54.434082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:54.455849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa76bdc-374f-40e8-b533-831ac370ad04 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:54.464104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0101f-e707-47f1-abfd-6f415d1d119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:54.465164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0101f-e707-47f1-abfd-6f415d1d119b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:54.465444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0101f-e707-47f1-abfd-6f415d1d119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:54.465957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0101f-e707-47f1-abfd-6f415d1d119b admin admin] PATH_INFO: `/v3/projects/6518b545f2ff43389a1aaee061d468ee/users/eee37b2b8a714aeda9cce8488d5672af/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:54.469003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0101f-e707-47f1-abfd-6f415d1d119b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6518b545f2ff43389a1aaee061d468ee, user_id=eee37b2b8a714aeda9cce8488d5672af, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:54.472967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0101f-e707-47f1-abfd-6f415d1d119b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:54.506590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 987/1987] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:54 2022] PUT /identity/v3/projects/6518b545f2ff43389a1aaee061d468ee/users/eee37b2b8a714aeda9cce8488d5672af/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:54.533291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:54.533761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:54.534197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:54.571366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b874ab0db696418ebcba39fd50bcaf24', 'request_id': 'req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:54.600237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b874ab0db696418ebcba39fd50bcaf24', 'request_id': 'req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:54.609073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] MFA Rules not processed for user `eee37b2b8a714aeda9cce8488d5672af`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:54.611801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:54.621690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b874ab0db696418ebcba39fd50bcaf24', 'request_id': 'req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:54.654142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] The token cache is being invalidate because project b874ab0db696418ebcba39fd50bcaf24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:54.654749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:54.659179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:54.674782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1001/1988] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:54 2022] DELETE /identity/v3/projects/b874ab0db696418ebcba39fd50bcaf24 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:54.683273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:54.688571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:54.857662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 988/1989] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:54.870091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:54.883111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:54.885300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:54.936776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db16f2-6cb1-4a1b-8932-2cb812c7f58d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:55.160132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82a96f-3152-45ff-a64a-e17a032de42a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:55.175309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:55.176937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:55.177433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:55.177936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:55.190548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:55.192051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:55.192514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:55.192980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] PATH_INFO: `/v3/users/41517dbe39fd4f51a1662cf5fad255e5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:55.204624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41517dbe39fd4f51a1662cf5fad255e5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:55.209824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:55.236580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:55.245768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:55.285576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 989/1990] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:27:55.374700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:55.377734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:55.452346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41517dbe39fd4f51a1662cf5fad255e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.485812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc49f3c-860c-4e3b-bc92-261daf6d8b9b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:55.501517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41517dbe39fd4f51a1662cf5fad255e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.522351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:55.527563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:55.528208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:55.528870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] PATH_INFO: `/v3/users/03f85d8fd69341549899cc1e8d9d5d7d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:55.533662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f85d8fd69341549899cc1e8d9d5d7d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:55.537647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41517dbe39fd4f51a1662cf5fad255e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.540304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41517dbe39fd4f51a1662cf5fad255e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.542377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41517dbe39fd4f51a1662cf5fad255e5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.547055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:55.618931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1002/1991] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:54 2022] DELETE /identity/v3/users/41517dbe39fd4f51a1662cf5fad255e5 => generated 0 bytes in 937 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:55.711118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:55.754276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f85d8fd69341549899cc1e8d9d5d7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.787253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f85d8fd69341549899cc1e8d9d5d7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.789067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f85d8fd69341549899cc1e8d9d5d7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.793469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f85d8fd69341549899cc1e8d9d5d7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.834769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f85d8fd69341549899cc1e8d9d5d7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:55.897744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 990/1992] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:55 2022] DELETE /identity/v3/users/03f85d8fd69341549899cc1e8d9d5d7d => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:55.927347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:55.929674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:56.128407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372641a5-afaa-4a25-a649-621cff87cb52 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:56.189719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:56.264728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:56.267081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:56.267763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:56.268293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:56.272010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:56.400250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:56.401833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:56.421108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1003/1993] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:27:56.506417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:56.509507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:56.608762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:56.614447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:56.725663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6eded-79c5-4bf8-8075-4d270682dbad admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:56.737889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:56.740903 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:56.741959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:56.742568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] PATH_INFO: `/v3/projects/3cf302baf3b7484da9af939e59af0abc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:56.744936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf302baf3b7484da9af939e59af0abc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:56.750804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:56.823574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b6ab0-61dd-43d9-8d87-33a55b0f90de service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:56.832547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:56.834202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:56.836714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:56.837197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] PATH_INFO: `/v3/projects/99154617513f43a7b78a5e861593083d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:56.843294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=99154617513f43a7b78a5e861593083d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:56.848267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:56.873133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf302baf3b7484da9af939e59af0abc', 'request_id': 'req-52522657-3fa2-49a3-9e16-d5396637eeb6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:56.884361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf302baf3b7484da9af939e59af0abc', 'request_id': 'req-52522657-3fa2-49a3-9e16-d5396637eeb6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:56.903515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf302baf3b7484da9af939e59af0abc', 'request_id': 'req-52522657-3fa2-49a3-9e16-d5396637eeb6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:56.940312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] The token cache is being invalidate because project 3cf302baf3b7484da9af939e59af0abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:56.941407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:56.943693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:56.947711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 991/1994] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:56 2022] DELETE /identity/v3/projects/3cf302baf3b7484da9af939e59af0abc => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:56.953760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:56.956372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:56.969492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99154617513f43a7b78a5e861593083d', 'request_id': 'req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.007113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99154617513f43a7b78a5e861593083d', 'request_id': 'req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.036266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99154617513f43a7b78a5e861593083d', 'request_id': 'req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.074619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] The token cache is being invalidate because project 99154617513f43a7b78a5e861593083d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:57.075921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.077550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f31e23fd-9345-4bf9-8ee8-7c791082cc1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.092388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1004/1995] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:56 2022] DELETE /identity/v3/projects/99154617513f43a7b78a5e861593083d => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:57.097870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151cad6-70e7-492d-b6d2-8b4984164c0b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:57.098641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151cad6-70e7-492d-b6d2-8b4984164c0b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:57.099241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151cad6-70e7-492d-b6d2-8b4984164c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:57.103951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:57.119168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:57.150281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6151cad6-70e7-492d-b6d2-8b4984164c0b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:57.152921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6151cad6-70e7-492d-b6d2-8b4984164c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:57.159817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52522657-3fa2-49a3-9e16-d5396637eeb6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:57.350697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1005/1996] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:57.354960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:57.356280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:57.356866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:57.357365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] PATH_INFO: `/v3/users/8e4f9f6b4473445da694a09483a2c193` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:57.360782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e4f9f6b4473445da694a09483a2c193)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:57.365203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:57.366299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:57.366759 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:57.367046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:57.393228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:57.395480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:57.411703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1006/1997] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:57.420690 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:57.438487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e846de-75e5-4e2b-9b77-18110051602d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:57.446367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:57.447553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:57.447972 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:57.449630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:57.449934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:57.450632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:57.453893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:57.461364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e4f9f6b4473445da694a09483a2c193'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.478582 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e4f9f6b4473445da694a09483a2c193'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.479806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e4f9f6b4473445da694a09483a2c193'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.481109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e4f9f6b4473445da694a09483a2c193'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.482783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1007/1998] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:57.484436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:57.487950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:57.495150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e4f9f6b4473445da694a09483a2c193'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:57.510753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ac3cb-a130-432d-9d6a-e5765672734b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:57.520191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:57.521852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:57.522193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:57.522583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:57.523320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:57.527044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:57.527951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 992/1999] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:27:56 2022] DELETE /identity/v3/users/8e4f9f6b4473445da694a09483a2c193 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:57.574900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1008/2000] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:57 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:57.580539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:57.730096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e5225-9800-4514-af9b-53b691b66cfe admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:57.772318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:57.900983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:57.902726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:57.903484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:57.904246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:57.906753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:57.910420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:57.962442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 993/2001] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:57 2022] POST /identity/v3/users => generated 284 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:57.968855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.004984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.007338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:58.049344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c13a90-354b-4eb2-89e1-8f7ed1b8ddc0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:58.059296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78716859-c4e5-4af2-9ace-a5cc0af4fbaa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:58.070098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:58.072216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.072216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.072634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] PATH_INFO: `/v3/projects/da140836842b40fdb1c4248ef367c0a8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.077971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da140836842b40fdb1c4248ef367c0a8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:58.085238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:58.087988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.089397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:58.089936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.092914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.094346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:58.102001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:58.125041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1009/2002] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:58.131766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.170903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518680f-6cf0-4192-82a8-0eb2fa5c92dc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:58.186574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:58.187236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.187521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.187805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.188651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:58.191273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:58.228505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1010/2003] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:58.232726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da140836842b40fdb1c4248ef367c0a8', 'request_id': 'req-61f67973-946d-4173-ac40-2240c75c11b8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:58.234918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.252814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da140836842b40fdb1c4248ef367c0a8', 'request_id': 'req-61f67973-946d-4173-ac40-2240c75c11b8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:58.267268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da140836842b40fdb1c4248ef367c0a8', 'request_id': 'req-61f67973-946d-4173-ac40-2240c75c11b8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:58.270281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b098ec-5c80-4143-8e08-51e3f49b6237 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:58.312063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] The token cache is being invalidate because project da140836842b40fdb1c4248ef367c0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:27:58.312820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:58.315597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-61f67973-946d-4173-ac40-2240c75c11b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:27:58.327482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 994/2004] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:27:58 2022] DELETE /identity/v3/projects/da140836842b40fdb1c4248ef367c0a8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:58.334691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc8aaa-7737-4939-8ae7-86c7c0f5310c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.335092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc8aaa-7737-4939-8ae7-86c7c0f5310c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.335476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc8aaa-7737-4939-8ae7-86c7c0f5310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.400049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bcfc8aaa-7737-4939-8ae7-86c7c0f5310c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:58.401901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bcfc8aaa-7737-4939-8ae7-86c7c0f5310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:58.446559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f26c4-f3dc-437f-aefb-cc97e491f995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:58.448530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f26c4-f3dc-437f-aefb-cc97e491f995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.449327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f26c4-f3dc-437f-aefb-cc97e491f995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.450184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f26c4-f3dc-437f-aefb-cc97e491f995 admin admin] PATH_INFO: `/v3/projects/61186bd62efd4bbfa20a9b8ac60552e1/users/7171127fae134a1e9ed001d9d0389928/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.490399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f26c4-f3dc-437f-aefb-cc97e491f995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61186bd62efd4bbfa20a9b8ac60552e1, user_id=7171127fae134a1e9ed001d9d0389928, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:58.496461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f26c4-f3dc-437f-aefb-cc97e491f995 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:58.543358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1011/2005] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:58 2022] PUT /identity/v3/projects/61186bd62efd4bbfa20a9b8ac60552e1/users/7171127fae134a1e9ed001d9d0389928/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:27:58.561015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.561604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.562070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.572184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 995/2006] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:58.587878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.588421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.588982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.627160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:58.630044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:58.631326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] MFA Rules not processed for user `7171127fae134a1e9ed001d9d0389928`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:58.635115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:58.661088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:58.720232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:58.783893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1012/2007] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:27:58.790390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.792671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:27:58.809447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 996/2008] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:27:58.811144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.820230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b6ada-6e3b-4d79-a953-11dd9a99d4e4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:58.829419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:58.830481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.830820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c113e3b-d96f-4434-bf1e-9759b4a9c26a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:58.831471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.831989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.833195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:58.836294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:58.844400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:58.845493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:58.846012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:58.846568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:58.847866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:27:58.848448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:58.851901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:58.889531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 997/2009] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:27:58.891489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.896527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1013/2010] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:58 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:58.901843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:58.914238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92165c75-b7be-4538-a7a2-34ef210a984f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.064566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.065618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861c873-b0ea-490e-905d-cc1ed3a13c9e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.066124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.067137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.067868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.069276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.072675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.073670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.074080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.074685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.075242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.077295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.086234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.144527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1014/2011] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:58 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:27:59.152428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 998/2012] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:27:58 2022] POST /identity/v3/projects => generated 375 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:59.154366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.157575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.187900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32c430-3cf5-43b3-ab79-968704c797b8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.325977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0dc30-d4e7-46b8-95e4-58ac9fef63d8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.336969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.338802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.339538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.340353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.342327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.344436 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.345145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.345439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.345722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.346485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.348262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.351553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.370968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1015/2013] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:59.378767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.396378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 999/2014] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:27:59 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:27:59.401916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b45ba7-e6d6-4af0-9efe-492ce9066ffa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.402730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.411224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.412881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.413567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.414057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] PATH_INFO: `/v3/projects/551e34d216414dd49a02f28d978db753/users/ae7d600c3ab14a90a528bffc5b5cb916/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.420121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca316cc-5650-4c9d-9f73-430e4bbfb72c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.427594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.429253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.430040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.430743 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.432282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.435222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.446384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551e34d216414dd49a02f28d978db753, user_id=ae7d600c3ab14a90a528bffc5b5cb916, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.452985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.461531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1000/2015] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:27:59.461781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:59.464579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.484049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1016/2016] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:59 2022] PUT /identity/v3/projects/551e34d216414dd49a02f28d978db753/users/ae7d600c3ab14a90a528bffc5b5cb916/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:59.488716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb936e6-66fc-464a-bf93-bc7580260404 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.490798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.546710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:27:59.611700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.612776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.613522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.613957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.615366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.618659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.649196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1001/2017] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:59.655802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.675556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dd813-1cb0-41f6-9db6-87a0b59a9fec admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.699244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d5c98-70a0-479e-9b82-0df2b9cd266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.701796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d5c98-70a0-479e-9b82-0df2b9cd266c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.702385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d5c98-70a0-479e-9b82-0df2b9cd266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.703066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d5c98-70a0-479e-9b82-0df2b9cd266c admin admin] PATH_INFO: `/v3/projects/71544a768d944b48ae276bffb40cdbd3/users/12edc1471e0c4293bc4147ddc3a87e6b/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.703248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca553067-32fa-4210-b95d-efb073e5c8a1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.734594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:27:59.737638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d5c98-70a0-479e-9b82-0df2b9cd266c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71544a768d944b48ae276bffb40cdbd3, user_id=12edc1471e0c4293bc4147ddc3a87e6b, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.739712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.740334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.742178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.742570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d5c98-70a0-479e-9b82-0df2b9cd266c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.743800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:27:59.753632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:27:59.778295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1002/2018] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:59 2022] PUT /identity/v3/projects/71544a768d944b48ae276bffb40cdbd3/users/12edc1471e0c4293bc4147ddc3a87e6b/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:27:59.789027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1017/2019] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:27:59.796400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:27:59.797440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:27:59.798084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:27:59.801469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:27:59.852665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabf40a-4c33-45f3-85b0-e2d591615c9d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:27:59.861542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] MFA Rules not processed for user `12edc1471e0c4293bc4147ddc3a87e6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:27:59.863366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:00.015554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990dca26-a0a7-496f-8a0f-f90beeb81879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.017658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dca26-a0a7-496f-8a0f-f90beeb81879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.019492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dca26-a0a7-496f-8a0f-f90beeb81879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.020017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990dca26-a0a7-496f-8a0f-f90beeb81879 admin admin] PATH_INFO: `/v3/projects/551e34d216414dd49a02f28d978db753/users/ae7d600c3ab14a90a528bffc5b5cb916/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.024488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990dca26-a0a7-496f-8a0f-f90beeb81879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551e34d216414dd49a02f28d978db753, user_id=ae7d600c3ab14a90a528bffc5b5cb916, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.031502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990dca26-a0a7-496f-8a0f-f90beeb81879 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.044531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1003/2020] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:27:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:00.048137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.055772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1018/2021] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:27:59 2022] PUT /identity/v3/projects/551e34d216414dd49a02f28d978db753/users/ae7d600c3ab14a90a528bffc5b5cb916/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:00.070836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fae075-f0ed-422a-a646-8b261defddbb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.075153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58946062-3e01-4451-b379-e3d060186c06 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.075887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58946062-3e01-4451-b379-e3d060186c06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.076127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58946062-3e01-4451-b379-e3d060186c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.127959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-58946062-3e01-4451-b379-e3d060186c06 None None] MFA Rules not processed for user `ae7d600c3ab14a90a528bffc5b5cb916`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:00.130250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58946062-3e01-4451-b379-e3d060186c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:00.195264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.196448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.196928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.197570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.198923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.203012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.259317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1004/2022] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:00 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:00.260809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.281036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:00.302744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1019/2023] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:00.318683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58946062-3e01-4451-b379-e3d060186c06 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.321923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58946062-3e01-4451-b379-e3d060186c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:00.343940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:00.405247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f64653-d299-468c-84eb-c7d3920a859d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.413985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.415147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.415564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.416025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.417231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.421139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.450639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58946062-3e01-4451-b379-e3d060186c06 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.452423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1005/2024] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:00 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:00.457029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.470029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ce3b5-108e-4644-a0d6-846b25c13f91 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.475918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.476614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.476902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.477184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.477931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.479608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.495826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1006/2025] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:00.507702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.520038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474c759-9ac2-4861-a82b-743ceee3e020 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.526554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.528024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.528592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.529210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] PATH_INFO: `/v3/projects/29b9098367b847db8f1aa79976a8d8d0/users/6ea6364d85704a23a3415c373970f53e/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.550155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.551228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.551228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.551528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.557252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b9098367b847db8f1aa79976a8d8d0, user_id=6ea6364d85704a23a3415c373970f53e, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.563304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.586096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1007/2026] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:00 2022] PUT /identity/v3/projects/29b9098367b847db8f1aa79976a8d8d0/users/6ea6364d85704a23a3415c373970f53e/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:00.592233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.640398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.642430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.655034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1020/2027] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:00.711140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bac6a-a066-4920-a28e-7a7e196542e8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.719920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c65a09-098a-4a7a-9ec6-a9bf0cba6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.721041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c65a09-098a-4a7a-9ec6-a9bf0cba6124 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.721512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c65a09-098a-4a7a-9ec6-a9bf0cba6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.721986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c65a09-098a-4a7a-9ec6-a9bf0cba6124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.723341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c65a09-098a-4a7a-9ec6-a9bf0cba6124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.726490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c65a09-098a-4a7a-9ec6-a9bf0cba6124 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.745968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1008/2028] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:00.753058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:00.773562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acfcbc-9381-42be-9b96-41cc95031c24 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:00.780282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:00.781300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.781803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.782177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] PATH_INFO: `/v3/projects/29b9098367b847db8f1aa79976a8d8d0/users/6ea6364d85704a23a3415c373970f53e/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.784698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b9098367b847db8f1aa79976a8d8d0, user_id=6ea6364d85704a23a3415c373970f53e, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:00.789000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:00.806561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1021/2029] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:00 2022] PUT /identity/v3/projects/29b9098367b847db8f1aa79976a8d8d0/users/6ea6364d85704a23a3415c373970f53e/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:00.825015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:00.825979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:00.826173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:00.877875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] MFA Rules not processed for user `6ea6364d85704a23a3415c373970f53e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:00.880015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:00.983401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:01.023517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1009/2030] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:01.037017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:01.039315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:01.178557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4f105-b8c2-4f85-a715-2250f2cabc24 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:01.186045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af0225-42df-4c58-858e-19b116982723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:01.187130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af0225-42df-4c58-858e-19b116982723 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:01.187871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af0225-42df-4c58-858e-19b116982723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:01.188337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af0225-42df-4c58-858e-19b116982723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:01.215041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:01.301982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af0225-42df-4c58-858e-19b116982723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:01.304355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af0225-42df-4c58-858e-19b116982723 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:01.319324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1022/2031] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:03.015424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:03.111328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1ab50-e0a9-4142-998a-0e2fdd9620d5 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:03.120326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:03.121482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:03.122005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:03.122439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] PATH_INFO: `/v3/users/7171127fae134a1e9ed001d9d0389928` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:03.124711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7171127fae134a1e9ed001d9d0389928)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:03.128196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:03.189416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7171127fae134a1e9ed001d9d0389928'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:03.206164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7171127fae134a1e9ed001d9d0389928'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:03.207032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7171127fae134a1e9ed001d9d0389928'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:03.207749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7171127fae134a1e9ed001d9d0389928'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:03.217756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7171127fae134a1e9ed001d9d0389928'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:03.239736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1010/2032] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:03 2022] DELETE /identity/v3/users/7171127fae134a1e9ed001d9d0389928 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:03.244512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:03.245331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:03.245509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:03.264061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:03.265305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:03.346968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1023/2033] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:03.358430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:03.360735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:03.440351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c0a97e-27f3-41f0-a5be-68911a3d5466 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:03.446309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:03.447682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:03.448194 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:03.448864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:03.466392 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:03.468847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:03.483602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1011/2034] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:03.934003 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:03.939485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:03.955491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1ba09-918a-44c8-885d-e09ca99ad57e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:03.967567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:03.968544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:03.968877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:03.969498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] PATH_INFO: `/v3/projects/61186bd62efd4bbfa20a9b8ac60552e1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:03.971105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=61186bd62efd4bbfa20a9b8ac60552e1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:03.974009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:04.061463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61186bd62efd4bbfa20a9b8ac60552e1', 'request_id': 'req-06101139-8302-4c7e-a4cd-db9058188d4c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.073010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61186bd62efd4bbfa20a9b8ac60552e1', 'request_id': 'req-06101139-8302-4c7e-a4cd-db9058188d4c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.081234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61186bd62efd4bbfa20a9b8ac60552e1', 'request_id': 'req-06101139-8302-4c7e-a4cd-db9058188d4c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.096863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] The token cache is being invalidate because project 61186bd62efd4bbfa20a9b8ac60552e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:04.097314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.098118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.099996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1024/2035] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:03 2022] DELETE /identity/v3/projects/61186bd62efd4bbfa20a9b8ac60552e1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:04.104485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:04.116129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a51d10-6239-4d75-9189-ea9fc9fe9913 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:04.120516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:04.206699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:04.208300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:04.208798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:04.209305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] PATH_INFO: `/v3/users/ae7d600c3ab14a90a528bffc5b5cb916` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:04.211575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7d600c3ab14a90a528bffc5b5cb916)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:04.215236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:04.307557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7d600c3ab14a90a528bffc5b5cb916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.320931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7d600c3ab14a90a528bffc5b5cb916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.321826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7d600c3ab14a90a528bffc5b5cb916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.322642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7d600c3ab14a90a528bffc5b5cb916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.333966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-880874ef-abd7-40d2-a0e2-35308d6a870c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7d600c3ab14a90a528bffc5b5cb916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.363999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1012/2036] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:04 2022] DELETE /identity/v3/users/ae7d600c3ab14a90a528bffc5b5cb916 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:04.749154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:04.751511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:04.771936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06101139-8302-4c7e-a4cd-db9058188d4c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:04.837604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:04.878862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:04.880426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:04.881033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:04.881536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] PATH_INFO: `/v3/projects/551e34d216414dd49a02f28d978db753` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:04.883980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] RBAC: Authorizing `identity:delete_project(project_id=551e34d216414dd49a02f28d978db753)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:04.887821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:04.978826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551e34d216414dd49a02f28d978db753', 'request_id': 'req-4ef4c622-a501-4994-8bee-f0e30966128b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:04.996245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551e34d216414dd49a02f28d978db753', 'request_id': 'req-4ef4c622-a501-4994-8bee-f0e30966128b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:05.010040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551e34d216414dd49a02f28d978db753', 'request_id': 'req-4ef4c622-a501-4994-8bee-f0e30966128b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:05.023918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] The token cache is being invalidate because project 551e34d216414dd49a02f28d978db753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:05.024233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:05.024901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ef4c622-a501-4994-8bee-f0e30966128b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:05.026550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1025/2037] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:04 2022] DELETE /identity/v3/projects/551e34d216414dd49a02f28d978db753 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:05.032788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.032788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.033242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.059944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:05.062131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:05.148726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1013/2038] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:05.160920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.160920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.161564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.181647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:05.182987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:05.192592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1026/2039] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:05.197571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.208764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3173953-067b-4189-8398-a8206bc18f04 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.214969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.216122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.216397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.216688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.217471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:05.217787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.219491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.234292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1014/2040] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:05.240035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.251740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723ed6a-ea8d-4608-ab3b-d4dc414bf669 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.263343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.264277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.264649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.265028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.265968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.268721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.313852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1027/2041] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:05.316267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.332516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f480155-51ed-40f8-b64f-c317d7bd17d2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.350343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:05.419373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.420094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.420387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.420668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.421401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.423516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.451937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1015/2042] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:05.459031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.478946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9151c9c-72f2-464f-af0e-1a605eb4730b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.487939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.489153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.489665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.490176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.491678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.494915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.516135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1028/2043] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:05.521688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.536517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231707b1-ec87-4895-b16d-9a031137a955 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.542463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f729e-936b-4205-aa96-f84f5d1cfc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.543190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f729e-936b-4205-aa96-f84f5d1cfc89 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.543543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f729e-936b-4205-aa96-f84f5d1cfc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.543904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f729e-936b-4205-aa96-f84f5d1cfc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.544829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f729e-936b-4205-aa96-f84f5d1cfc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.546573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f729e-936b-4205-aa96-f84f5d1cfc89 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.571658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1016/2044] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:05.576741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.593636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b0d9d-7aaf-4b7d-a78a-4e9c709e9394 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.600115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.601291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.601624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.602055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] PATH_INFO: `/v3/projects/7f358727f7764eeda89b32c1eeeb9a53/users/04c7f4631c6b4068a7c64e6487ceb400/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.628029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f358727f7764eeda89b32c1eeeb9a53, user_id=04c7f4631c6b4068a7c64e6487ceb400, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.633033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.661760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1029/2045] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:05 2022] PUT /identity/v3/projects/7f358727f7764eeda89b32c1eeeb9a53/users/04c7f4631c6b4068a7c64e6487ceb400/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:05.680626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.681139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.681665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.724157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] MFA Rules not processed for user `04c7f4631c6b4068a7c64e6487ceb400`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:05.726195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:05.807961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1017/2046] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:05.813636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:05.914749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f5056-21b6-4500-8a2d-56001fa60f69 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:05.926578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:05.927284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:05.927565 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:05.927846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:05.928581 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:05.930593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:05.945794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:05.976903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1030/2047] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:05.985291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:06.003613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dde38f-65a5-4f2e-bc25-10d2c639f5ae None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:06.100042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:06.101377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.101895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.102380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.104037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:06.107464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:06.132486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1018/2048] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:05 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:06.137445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:06.150883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05007139-9f52-4f5b-b842-9d66ed92a152 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:06.158655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:06.159511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.159810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.160146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.161214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:06.163021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:06.184602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1031/2049] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:06.190068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:06.211439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ed41b-7131-420a-a6fb-a8c1e6b9a6f1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:06.218218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:06.218218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.218218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.218218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] PATH_INFO: `/v3/projects/02feaac6b171495fbf9c91b03ffddbb7/users/47b658e13bf644da994d92cfb55476e6/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.238338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:06.252665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02feaac6b171495fbf9c91b03ffddbb7, user_id=47b658e13bf644da994d92cfb55476e6, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:06.261188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:06.285862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1019/2050] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:06 2022] PUT /identity/v3/projects/02feaac6b171495fbf9c91b03ffddbb7/users/47b658e13bf644da994d92cfb55476e6/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:06.291987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:06.306123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09b62-f751-47d8-a36f-e06b1a609689 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:06.415718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa34cf3-991c-41cb-a932-f024c3406c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:06.416746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa34cf3-991c-41cb-a932-f024c3406c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.417214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa34cf3-991c-41cb-a932-f024c3406c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.417689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa34cf3-991c-41cb-a932-f024c3406c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.419198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa34cf3-991c-41cb-a932-f024c3406c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:06.422222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa34cf3-991c-41cb-a932-f024c3406c2d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:06.443169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1032/2051] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:06.449817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:06.470658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3b7e9-967d-44c0-b3ed-ffc53006c58b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:06.478816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:06.480484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.481072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.481555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] PATH_INFO: `/v3/projects/02feaac6b171495fbf9c91b03ffddbb7/users/47b658e13bf644da994d92cfb55476e6/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.485298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02feaac6b171495fbf9c91b03ffddbb7, user_id=47b658e13bf644da994d92cfb55476e6, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:06.490755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:06.510416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1020/2052] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:06 2022] PUT /identity/v3/projects/02feaac6b171495fbf9c91b03ffddbb7/users/47b658e13bf644da994d92cfb55476e6/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:06.529771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.530248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.530768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.577066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] MFA Rules not processed for user `47b658e13bf644da994d92cfb55476e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:06.579499 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:06.724052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1033/2053] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:06.742349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:06.745325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:06.850783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297462c2-39cc-4595-a5aa-a8599b164ac5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:06.857315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d73f2-446b-4140-8223-f3e2c9fc15fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:06.858309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d73f2-446b-4140-8223-f3e2c9fc15fb service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:06.858746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d73f2-446b-4140-8223-f3e2c9fc15fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:06.859252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d73f2-446b-4140-8223-f3e2c9fc15fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:06.872833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d73f2-446b-4140-8223-f3e2c9fc15fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:06.875696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d73f2-446b-4140-8223-f3e2c9fc15fb service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:06.895357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1021/2054] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:06.940657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:07.045893 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747afa7-38e3-4243-bfba-cb4ad3fdf719 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:07.051351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:07.052082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:07.052364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:07.052790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] PATH_INFO: `/v3/users/04c7f4631c6b4068a7c64e6487ceb400` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:07.053964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c7f4631c6b4068a7c64e6487ceb400)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:07.056154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:07.060600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:07.137376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c7f4631c6b4068a7c64e6487ceb400'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:07.149490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c7f4631c6b4068a7c64e6487ceb400'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:07.161183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c7f4631c6b4068a7c64e6487ceb400'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:07.162262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c7f4631c6b4068a7c64e6487ceb400'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:07.163274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c7f4631c6b4068a7c64e6487ceb400'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:07.196188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1034/2055] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:06 2022] DELETE /identity/v3/users/04c7f4631c6b4068a7c64e6487ceb400 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:07.202415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:07.203249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:07.203249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:07.230669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:07.233163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:07.367065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1022/2056] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:07.384444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:07.387732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:07.537368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c19d3e-14a0-4224-8342-f690e30a35ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:07.553411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:07.555439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:07.556122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:07.557008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:07.590390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:07.594013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:07.614244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1035/2057] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:08.165769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:08.168014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:08.181551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbd90b-9e10-43bb-94da-5406e6d1910a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:08.187315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:08.188415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:08.188798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:08.189192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] PATH_INFO: `/v3/projects/7f358727f7764eeda89b32c1eeeb9a53` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:08.190720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f358727f7764eeda89b32c1eeeb9a53)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:08.192858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:08.256109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f358727f7764eeda89b32c1eeeb9a53', 'request_id': 'req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.263503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f358727f7764eeda89b32c1eeeb9a53', 'request_id': 'req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.274564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f358727f7764eeda89b32c1eeeb9a53', 'request_id': 'req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.289967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:08.300638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] The token cache is being invalidate because project 7f358727f7764eeda89b32c1eeeb9a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:08.301260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.302646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.305264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1023/2058] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:08 2022] DELETE /identity/v3/projects/7f358727f7764eeda89b32c1eeeb9a53 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:08.310096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:08.327645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e863c-95a7-45de-892f-60685fc7cba3 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:08.426155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:08.427046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:08.427327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:08.427599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] PATH_INFO: `/v3/users/47b658e13bf644da994d92cfb55476e6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:08.429155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b658e13bf644da994d92cfb55476e6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:08.431146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:08.467261 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:08.508272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b658e13bf644da994d92cfb55476e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.520239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b658e13bf644da994d92cfb55476e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.534171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b658e13bf644da994d92cfb55476e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.535851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b658e13bf644da994d92cfb55476e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.536313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b658e13bf644da994d92cfb55476e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:08.572601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1036/2059] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:08 2022] DELETE /identity/v3/users/47b658e13bf644da994d92cfb55476e6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:08.839313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:08.840766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:08.857328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb5b51-8370-4cb2-a661-f3f17fd91e81 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:08.930031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:08.931072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:08.931454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:08.931914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] PATH_INFO: `/v3/users/12edc1471e0c4293bc4147ddc3a87e6b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:08.934806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=12edc1471e0c4293bc4147ddc3a87e6b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:08.939683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:09.031073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12edc1471e0c4293bc4147ddc3a87e6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.040396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12edc1471e0c4293bc4147ddc3a87e6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.041408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12edc1471e0c4293bc4147ddc3a87e6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.042374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12edc1471e0c4293bc4147ddc3a87e6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.057280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-4591e620-f601-4dad-a426-36ee0b67c91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12edc1471e0c4293bc4147ddc3a87e6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.088116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:09.091732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:09.099033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1024/2060] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:08 2022] DELETE /identity/v3/users/12edc1471e0c4293bc4147ddc3a87e6b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:09.105167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.106603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.106603 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.130082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a932d9f-bd01-4a44-b5f8-55d0d4abd639 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:09.143573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:09.145688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:09.246867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1025/2061] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:09.249807 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:09.251334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.251824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.252341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] PATH_INFO: `/v3/projects/02feaac6b171495fbf9c91b03ffddbb7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.254330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=02feaac6b171495fbf9c91b03ffddbb7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:09.257647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:09.261546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:09.263520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:09.313259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:09.361564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02feaac6b171495fbf9c91b03ffddbb7', 'request_id': 'req-dfb4c135-4d50-4d42-b149-48190eae83ec'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.376491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02feaac6b171495fbf9c91b03ffddbb7', 'request_id': 'req-dfb4c135-4d50-4d42-b149-48190eae83ec'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.376784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bbc63-7a88-4f5e-a657-57d0a8d127da None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:09.388537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02feaac6b171495fbf9c91b03ffddbb7', 'request_id': 'req-dfb4c135-4d50-4d42-b149-48190eae83ec'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.410269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] The token cache is being invalidate because project 02feaac6b171495fbf9c91b03ffddbb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:09.410771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.412037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dfb4c135-4d50-4d42-b149-48190eae83ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:09.413950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1037/2062] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:09 2022] DELETE /identity/v3/projects/02feaac6b171495fbf9c91b03ffddbb7 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:09.424438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea78dd-589d-4c10-9788-30ea56ee2de3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.425283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea78dd-589d-4c10-9788-30ea56ee2de3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.425831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea78dd-589d-4c10-9788-30ea56ee2de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.447559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-14ea78dd-589d-4c10-9788-30ea56ee2de3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:09.448806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-14ea78dd-589d-4c10-9788-30ea56ee2de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:09.459885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:09.461000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.461000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.461300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.463429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:09.484242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:09.550961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1038/2063] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:09.556120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.556980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.557541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.558591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:09.562049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:09.574111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1026/2064] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:09.586369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:09.587685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:09.603877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1039/2065] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:09.607748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:09.635216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa639f5b-75ed-451d-8193-9dd1c1674292 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:09.646326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:09.647159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.647636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.647981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.648916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:09.649374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:09.654584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:09.675652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1027/2066] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:09.681415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:09.698266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce3607-a7e4-4f58-8593-a2941bc30bd9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:09.706966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:09.708354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.708994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.709455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.710847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:09.714152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:09.768804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1040/2067] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:09 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:09.773348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:09.795207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e918457-07ac-417f-a79d-91f37e0103a7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:09.908023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:09.909171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.909595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.910073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.911493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:09.919279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:09.958718 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1028/2068] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:09 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:09.965121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:09.980304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e136b-fa88-464c-9366-c4938406bbf5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:09.987817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:09.988977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:09.989464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:09.989943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:09.991346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:09.994890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.013550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1041/2069] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:10.018335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:10.025896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:10.032673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3783-d9a0-4b57-8a2d-7f0d168a3a78 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:10.042400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:10.043634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.044137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.044593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.045350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:10.047030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.059788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1029/2070] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:10.067224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:10.068221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:10.070437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:10.088273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067aa6d-37e6-456d-8967-877107a01795 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:10.096186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179be902-4a6a-44e5-8707-197e98458805 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:10.097371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e2d3e-5bc7-4e20-a6b2-c372b3eb613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:10.102280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:10.102903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2d3e-5bc7-4e20-a6b2-c372b3eb613a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.103453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2d3e-5bc7-4e20-a6b2-c372b3eb613a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.103634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.103934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2d3e-5bc7-4e20-a6b2-c372b3eb613a admin admin] PATH_INFO: `/v3/projects/6ccfde95485141febad498eb7f8df7a3/users/7b0ddc2bf6d14f6c812ebe842b65c98c/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.104136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.104403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] PATH_INFO: `/v3/projects/71544a768d944b48ae276bffb40cdbd3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.105964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=71544a768d944b48ae276bffb40cdbd3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:10.108213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.138874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e2d3e-5bc7-4e20-a6b2-c372b3eb613a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccfde95485141febad498eb7f8df7a3, user_id=7b0ddc2bf6d14f6c812ebe842b65c98c, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:10.141802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e2d3e-5bc7-4e20-a6b2-c372b3eb613a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.173144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1042/2071] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:10 2022] PUT /identity/v3/projects/6ccfde95485141febad498eb7f8df7a3/users/7b0ddc2bf6d14f6c812ebe842b65c98c/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:10.190811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.191368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.191989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.214281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71544a768d944b48ae276bffb40cdbd3', 'request_id': 'req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.223046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71544a768d944b48ae276bffb40cdbd3', 'request_id': 'req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.231474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71544a768d944b48ae276bffb40cdbd3', 'request_id': 'req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.232812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] MFA Rules not processed for user `7b0ddc2bf6d14f6c812ebe842b65c98c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:10.234642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:10.256108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] The token cache is being invalidate because project 71544a768d944b48ae276bffb40cdbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:10.256696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.257773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.262588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1030/2072] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:10 2022] DELETE /identity/v3/projects/71544a768d944b48ae276bffb40cdbd3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:10.266916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:10.269805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:10.351733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1043/2073] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:10.383629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:10.387037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:10.402473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b736e55-7b1a-4527-82a9-9bd3af1e69bc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:10.410780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:10.413405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.413944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.414243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] PATH_INFO: `/v3/users/6ea6364d85704a23a3415c373970f53e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.415834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea6364d85704a23a3415c373970f53e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:10.419863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.495766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11693e0e-8a0a-4a44-b4e5-a8d80ff660c3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:10.502502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:10.503831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.504402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.504902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.508700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea6364d85704a23a3415c373970f53e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.521264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea6364d85704a23a3415c373970f53e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.522060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:10.522294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea6364d85704a23a3415c373970f53e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.523240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea6364d85704a23a3415c373970f53e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.524317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.535578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b840b9be-ff32-4d74-add5-4ba965afc2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea6364d85704a23a3415c373970f53e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:10.538165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1044/2074] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:10.557785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:10.575613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1031/2075] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:10 2022] DELETE /identity/v3/users/6ea6364d85704a23a3415c373970f53e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:10.699374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:10.702471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:10.717774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:10.726938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f1e-e98c-4e0d-a276-749a9cd29d13 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:10.848265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:10.850265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:10.850794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:10.851160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] PATH_INFO: `/v3/users/790ca005d18a4424841f7fb88a7550bd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:10.853137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=790ca005d18a4424841f7fb88a7550bd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:10.856675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:10.972466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790ca005d18a4424841f7fb88a7550bd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.011007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790ca005d18a4424841f7fb88a7550bd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.027172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790ca005d18a4424841f7fb88a7550bd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.029009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790ca005d18a4424841f7fb88a7550bd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.031191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790ca005d18a4424841f7fb88a7550bd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.077158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1045/2076] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:10 2022] DELETE /identity/v3/users/790ca005d18a4424841f7fb88a7550bd => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:11.084567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.085066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.085569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.099877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.103699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:11.157617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:11.159364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:11.259250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1f8dc7-0db0-43ec-9215-92f7d585adbb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:11.267018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:11.269457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.269902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.270283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] PATH_INFO: `/v3/projects/29b9098367b847db8f1aa79976a8d8d0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.271883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b9098367b847db8f1aa79976a8d8d0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:11.274101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:11.281995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1032/2077] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:11.297647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.300755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:11.316841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:11.366032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b9098367b847db8f1aa79976a8d8d0', 'request_id': 'req-700b21cb-c48c-4005-8cd6-1d956a9b25f5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.383441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b9098367b847db8f1aa79976a8d8d0', 'request_id': 'req-700b21cb-c48c-4005-8cd6-1d956a9b25f5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.393785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b9098367b847db8f1aa79976a8d8d0', 'request_id': 'req-700b21cb-c48c-4005-8cd6-1d956a9b25f5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.410097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] The token cache is being invalidate because project 29b9098367b847db8f1aa79976a8d8d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:11.411026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.412204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-700b21cb-c48c-4005-8cd6-1d956a9b25f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:11.415088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1046/2078] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:11 2022] DELETE /identity/v3/projects/29b9098367b847db8f1aa79976a8d8d0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:11.421386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc382a3d-94e7-47f7-8cdb-27b5bbfc0d62 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.422588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc382a3d-94e7-47f7-8cdb-27b5bbfc0d62 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.422950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc382a3d-94e7-47f7-8cdb-27b5bbfc0d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.426565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7ed1d-0deb-4b03-92b9-6a793045e4c4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:11.458680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cc382a3d-94e7-47f7-8cdb-27b5bbfc0d62 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:11.460486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc382a3d-94e7-47f7-8cdb-27b5bbfc0d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:11.497732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:11.498484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.498796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.499071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.501573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:11.587592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1047/2079] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:11.600824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.601645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.602042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.625561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:11.627067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:11.631334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:11.632632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:11.633953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:11.645263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1048/2080] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:11.645859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1033/2081] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:11.649464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.662696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a3c3e-a2e4-4cfb-a197-4e03d257ceea None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:11.671287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:11.672515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.672813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.673377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.674265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:11.674660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:11.676447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:11.691652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1049/2082] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:11.697319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.709341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ed5a5-68ce-4a5d-a3e7-488e08bf5520 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:11.715488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:11.715488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.715929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.715929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.717129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:11.718775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:11.752090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1034/2083] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:11 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:11.756801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.772097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881759a1-f1cd-4e0a-a5bc-37c5d1e6b0dd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:11.877193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:11.877895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.878175 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.878494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.880000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:11.882312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:11.919772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1050/2084] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:11 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:11.921142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.934915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cf310-6ef9-4298-874c-e5b05d12146f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:11.951255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:11.952117 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:11.952409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:11.952698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:11.953649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:11.955815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:11.974571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1035/2085] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:11.977724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:11.991716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f342b-57f1-4ad4-a984-ce9c4c75a9df admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:12.000131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:12.001391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:12.001950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:12.002447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:12.003604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:12.006186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:12.025663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1051/2086] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:12.031095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:12.047544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdae556-527b-4c30-8d69-d51b9fd24606 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:12.055210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a8cb5-e912-4feb-ba57-2daac42abf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:12.056294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a8cb5-e912-4feb-ba57-2daac42abf65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:12.056744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a8cb5-e912-4feb-ba57-2daac42abf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:12.057222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a8cb5-e912-4feb-ba57-2daac42abf65 admin admin] PATH_INFO: `/v3/projects/310c69845a264bd4a88ec5c924d91d1b/users/a6ccfd18a84349728c002db30d75a5c0/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:12.085773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a8cb5-e912-4feb-ba57-2daac42abf65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310c69845a264bd4a88ec5c924d91d1b, user_id=a6ccfd18a84349728c002db30d75a5c0, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:12.091514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a8cb5-e912-4feb-ba57-2daac42abf65 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:12.112128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1036/2087] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:12 2022] PUT /identity/v3/projects/310c69845a264bd4a88ec5c924d91d1b/users/a6ccfd18a84349728c002db30d75a5c0/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:12.117091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:12.130573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:12.130917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:12.131385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:12.146182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:12.187082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] MFA Rules not processed for user `a6ccfd18a84349728c002db30d75a5c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:12.189529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:12.194371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504df1d-0f5d-42f0-9a43-8b44d532be73 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:12.218474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:12.304756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1037/2088] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:12.322230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:12.323549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:12.323668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:12.323965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:12.324698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] PATH_INFO: `/v3/projects/af907554b4264e3e8fc4c9014fff066f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:12.326454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:12.327064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af907554b4264e3e8fc4c9014fff066f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:12.331118 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:12.441111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af907554b4264e3e8fc4c9014fff066f', 'request_id': 'req-b29c10f4-422d-4f89-9446-e5dad57dbe46'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.455368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af907554b4264e3e8fc4c9014fff066f', 'request_id': 'req-b29c10f4-422d-4f89-9446-e5dad57dbe46'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.457170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184c551-a2d2-4d45-92be-307d24324671 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:12.467864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af907554b4264e3e8fc4c9014fff066f', 'request_id': 'req-b29c10f4-422d-4f89-9446-e5dad57dbe46'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.488347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] The token cache is being invalidate because project af907554b4264e3e8fc4c9014fff066f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:12.489171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.489939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.497768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1052/2089] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:12 2022] DELETE /identity/v3/projects/af907554b4264e3e8fc4c9014fff066f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:12.499191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:12.502162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:12.536574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:12.546415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:12.547671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:12.548621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:12.549327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:12.551378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:12.620285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c10f4-422d-4f89-9446-e5dad57dbe46 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:12.627514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:12.629178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:12.629668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:12.630179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] PATH_INFO: `/v3/users/eee37b2b8a714aeda9cce8488d5672af` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:12.633380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee37b2b8a714aeda9cce8488d5672af)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:12.637755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:12.647614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:12.650430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:12.679441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1038/2090] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:12.809409 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:12.817621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee37b2b8a714aeda9cce8488d5672af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.830462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee37b2b8a714aeda9cce8488d5672af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.863208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee37b2b8a714aeda9cce8488d5672af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.864037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee37b2b8a714aeda9cce8488d5672af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.864968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4ffa9a5a-a7d0-413e-9780-cf1b40d85dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee37b2b8a714aeda9cce8488d5672af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:12.917193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1053/2091] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:12 2022] DELETE /identity/v3/users/eee37b2b8a714aeda9cce8488d5672af => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:13.643863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:13.672101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e428c7-79cd-4bd0-95f0-b0a962d248a9 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:13.805700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:13.807928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:13.808878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:13.809413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] PATH_INFO: `/v3/projects/6518b545f2ff43389a1aaee061d468ee` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:13.811497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6518b545f2ff43389a1aaee061d468ee)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:13.815163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:13.911863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6518b545f2ff43389a1aaee061d468ee', 'request_id': 'req-5f42de55-ba29-460b-9aea-8d538ccbc51e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:13.920820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6518b545f2ff43389a1aaee061d468ee', 'request_id': 'req-5f42de55-ba29-460b-9aea-8d538ccbc51e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:13.932848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6518b545f2ff43389a1aaee061d468ee', 'request_id': 'req-5f42de55-ba29-460b-9aea-8d538ccbc51e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:13.976630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] The token cache is being invalidate because project 6518b545f2ff43389a1aaee061d468ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:13.977067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:13.978066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5f42de55-ba29-460b-9aea-8d538ccbc51e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:13.980330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1039/2092] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:13 2022] DELETE /identity/v3/projects/6518b545f2ff43389a1aaee061d468ee => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:13.987547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:13.988168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:13.988262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.009899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:14.011186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:14.085805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1054/2093] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:14.098372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.098910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.099423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.121719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:14.123004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:14.131255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1040/2094] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:14.135940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.148201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0c9ed-1f3c-4000-b739-bdbdd7d5e819 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.153326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.154008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.154440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.154753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.155507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:14.155819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.157464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.171454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1055/2095] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:14.176476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.187898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595be30-8288-40e7-88e6-fccf55718212 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.195526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.196487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.196793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.197177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.198899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.204347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.260736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1041/2096] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:14 2022] POST /identity/v3/projects => generated 367 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:14.263627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.298079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32f576-b8a2-459c-9d29-caa4c3f3a381 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.404468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.406905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.407547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.408153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.409507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.413537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.456146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1056/2097] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:14 2022] POST /identity/v3/users => generated 284 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:14.461208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.474368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:14.480489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4e517-b384-4601-a222-0ebd48c76ced admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.492052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.493331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.493933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.494172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.494977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.498297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.512166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1042/2098] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:14.517228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.532384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a061a-6e57-4dff-826b-7472f447b140 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.539720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c4498-784c-4fd7-ba72-57a170c0e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.540793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4498-784c-4fd7-ba72-57a170c0e08d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.541282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4498-784c-4fd7-ba72-57a170c0e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.541766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c4498-784c-4fd7-ba72-57a170c0e08d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.543342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c4498-784c-4fd7-ba72-57a170c0e08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.547413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c4498-784c-4fd7-ba72-57a170c0e08d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.567111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1057/2099] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:14.572914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.584562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ccf4a-2faf-4d63-afc7-eba4f423b56f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.589997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.591139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.591558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.591822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] PATH_INFO: `/v3/projects/5d8da26390f44da1bf01aeca22d6a26b/users/ad591b13df894fb39e192b0278d8f224/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.614841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8da26390f44da1bf01aeca22d6a26b, user_id=ad591b13df894fb39e192b0278d8f224, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.617510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.635215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1043/2100] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:14 2022] PUT /identity/v3/projects/5d8da26390f44da1bf01aeca22d6a26b/users/ad591b13df894fb39e192b0278d8f224/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:14.654338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.654974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.655092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.700808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] MFA Rules not processed for user `ad591b13df894fb39e192b0278d8f224`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:14.702378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:14.717662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:14.804942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1058/2101] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:14.806187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.896276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f5b06-608e-4876-9fc9-cf124058ff62 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:14.901785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:14.902698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:14.902995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:14.903285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:14.904079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:14.906309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:14.942995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1044/2102] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:14 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:14.947895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:14.964785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bde2c-7e22-4adf-b99b-f4fb1aa29371 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.066396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.067532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.068049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.068562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.069789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:15.073614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:15.111767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1059/2103] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:14 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:15.119129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:15.136576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c556408-f155-4131-a092-07e74d241c1b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.143917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.145090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.145591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.146058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.147251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:15.149254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:15.166921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1045/2104] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:15.171669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:15.186346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79bc399-684d-4e99-8bd4-f2fa4a5708cb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.194568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.196162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.196651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.197297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] PATH_INFO: `/v3/projects/63436d262b0c4871b7568b09940b544e/users/cbfbb4c5c83a4a14ba83d8c086b8b1a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.226016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63436d262b0c4871b7568b09940b544e, user_id=cbfbb4c5c83a4a14ba83d8c086b8b1a8, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:15.231734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:15.265364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1060/2105] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:15 2022] PUT /identity/v3/projects/63436d262b0c4871b7568b09940b544e/users/cbfbb4c5c83a4a14ba83d8c086b8b1a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:15.275972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:15.292851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01e0dc-3577-4a70-b72f-29b46fd29b4d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.387291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9321a-4e75-43ac-80aa-08845f5c0c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.388383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9321a-4e75-43ac-80aa-08845f5c0c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.388843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9321a-4e75-43ac-80aa-08845f5c0c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.389300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9321a-4e75-43ac-80aa-08845f5c0c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.390654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9321a-4e75-43ac-80aa-08845f5c0c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:15.393904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9321a-4e75-43ac-80aa-08845f5c0c83 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:15.406851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1046/2106] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:15.411961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:15.423793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32aec04-488d-46e7-81d2-17a882098260 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.430407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.431835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.432159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.432591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] PATH_INFO: `/v3/projects/63436d262b0c4871b7568b09940b544e/users/cbfbb4c5c83a4a14ba83d8c086b8b1a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.436319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63436d262b0c4871b7568b09940b544e, user_id=cbfbb4c5c83a4a14ba83d8c086b8b1a8, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:15.439531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:15.474976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1061/2107] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:15 2022] PUT /identity/v3/projects/63436d262b0c4871b7568b09940b544e/users/cbfbb4c5c83a4a14ba83d8c086b8b1a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:15.485653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.486315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.486980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.538172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] MFA Rules not processed for user `cbfbb4c5c83a4a14ba83d8c086b8b1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:15.540167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:15.595178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:15.652479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1047/2108] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:15.664910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:15.666829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:15.764878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925f04a-9651-42fe-8972-2cedb52c758b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.771313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.772165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.772683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.773155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:15.793506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:15.796066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:15.815607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1062/2109] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:15.934266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:15.961229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17261053-fa70-4503-8b91-a76ea6a4d9cb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:15.968834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6da33-b645-47bb-90fc-ab4c88ea581b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:15.970106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6da33-b645-47bb-90fc-ab4c88ea581b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:15.972549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6da33-b645-47bb-90fc-ab4c88ea581b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:15.972549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6da33-b645-47bb-90fc-ab4c88ea581b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:16.059300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6da33-b645-47bb-90fc-ab4c88ea581b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:16.062187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6da33-b645-47bb-90fc-ab4c88ea581b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:16.077846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1048/2110] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:21.345503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:21.347170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:21.364141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:21.441266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ccee2-0fc0-44d6-b677-b494e0b744cf service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:21.445992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:21.446924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:21.447212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:21.447485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] PATH_INFO: `/v3/users/a6ccfd18a84349728c002db30d75a5c0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:21.448942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ccfd18a84349728c002db30d75a5c0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:21.451054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:21.531187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ccfd18a84349728c002db30d75a5c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:21.542964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ccfd18a84349728c002db30d75a5c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:21.556263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ccfd18a84349728c002db30d75a5c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:21.557223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ccfd18a84349728c002db30d75a5c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:21.558354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ccfd18a84349728c002db30d75a5c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:21.591399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1063/2111] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:21 2022] DELETE /identity/v3/users/a6ccfd18a84349728c002db30d75a5c0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:21.597675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:21.598425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:21.598425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:21.618807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:21.620378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:21.705503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1049/2112] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:21.720246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:21.722057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:21.798163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a72346-37c7-40a1-887b-f39c341bb697 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:21.802117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab8f34-f6e3-4bc9-9e07-ce7487543791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:21.802818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab8f34-f6e3-4bc9-9e07-ce7487543791 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:21.803106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab8f34-f6e3-4bc9-9e07-ce7487543791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:21.803378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab8f34-f6e3-4bc9-9e07-ce7487543791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:21.813595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab8f34-f6e3-4bc9-9e07-ce7487543791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:21.815151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab8f34-f6e3-4bc9-9e07-ce7487543791 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:21.829463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1064/2113] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:22.214151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.217886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:22.252291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205548a-c19b-48f1-a40b-0c24852f673d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.289396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.289396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.289396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.289396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] PATH_INFO: `/v3/projects/310c69845a264bd4a88ec5c924d91d1b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.291019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310c69845a264bd4a88ec5c924d91d1b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.295051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.373526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310c69845a264bd4a88ec5c924d91d1b', 'request_id': 'req-c45b97f2-1b9f-4c60-b778-c199bd319889'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:22.384469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310c69845a264bd4a88ec5c924d91d1b', 'request_id': 'req-c45b97f2-1b9f-4c60-b778-c199bd319889'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:22.389770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:22.392601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310c69845a264bd4a88ec5c924d91d1b', 'request_id': 'req-c45b97f2-1b9f-4c60-b778-c199bd319889'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:22.411576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] The token cache is being invalidate because project 310c69845a264bd4a88ec5c924d91d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:22.411940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:22.412733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c45b97f2-1b9f-4c60-b778-c199bd319889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:22.414867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1050/2114] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:22 2022] DELETE /identity/v3/projects/310c69845a264bd4a88ec5c924d91d1b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:22.422330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.422976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.422976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.448204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:22.449990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:22.485516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:22.535356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1065/2115] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:22.546220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.546907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.547551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.577973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:22.579316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:22.588641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1051/2116] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:22.596471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.609979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be49ac-0214-4202-9eb1-8be9f5ba1af6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.615137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.615853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.616158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.616433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.617253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:22.617562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.619397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.633505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1066/2117] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:22.639076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.649560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74251846-f5cb-4ec1-9797-3ec9dd7b75b2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.656108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.656793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.657072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.657346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.658058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.660001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.689988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1052/2118] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:22 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:22.694875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.706238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4b09c-10b1-45e4-a176-13f996ed07b3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.778372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.779085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.779370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.779653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.780405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.782367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.809614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1067/2119] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:22 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:22.815237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.826882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3a37a-0376-42f7-9599-373befbcbea2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.834803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.835784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.836261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.836699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.837918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.840945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.858931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1053/2120] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:22.864574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.876456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4319d-deb6-4f77-ba8c-de2f7275066b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.881755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db5708-7348-4c3c-8bdf-6ed4e4988dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.882447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db5708-7348-4c3c-8bdf-6ed4e4988dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.882768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db5708-7348-4c3c-8bdf-6ed4e4988dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.883034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db5708-7348-4c3c-8bdf-6ed4e4988dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.883822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db5708-7348-4c3c-8bdf-6ed4e4988dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.885616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db5708-7348-4c3c-8bdf-6ed4e4988dcd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.898680 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1068/2121] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:22.903576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:22.916546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca969f22-bee2-41c8-8103-ceca63f3eca2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:22.922326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:22.923277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.923612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.923891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] PATH_INFO: `/v3/projects/d9d3a797588e467aaca0a843a39d7ae1/users/260ee759924245a1bbc198f24aca76fc/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:22.943567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d3a797588e467aaca0a843a39d7ae1, user_id=260ee759924245a1bbc198f24aca76fc, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:22.946272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:22.964995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1054/2122] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:22 2022] PUT /identity/v3/projects/d9d3a797588e467aaca0a843a39d7ae1/users/260ee759924245a1bbc198f24aca76fc/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:22.985494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:22.986004 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:22.986462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:23.026794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] MFA Rules not processed for user `260ee759924245a1bbc198f24aca76fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:23.028065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:23.113090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1069/2123] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:23.129401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:23.131300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:23.234394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1a4b7-bd74-4bac-855a-d761c255d026 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:23.238550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf56b9-0cdd-4201-85c0-2a45801e4c79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:23.239211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf56b9-0cdd-4201-85c0-2a45801e4c79 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:23.239489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf56b9-0cdd-4201-85c0-2a45801e4c79 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:23.239760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf56b9-0cdd-4201-85c0-2a45801e4c79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:23.255720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf56b9-0cdd-4201-85c0-2a45801e4c79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:23.257875 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf56b9-0cdd-4201-85c0-2a45801e4c79 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:23.274222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1055/2124] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:28.218850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:28.220702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:28.270127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:28.326431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59676f-c3ce-4688-916e-39abc4a0df58 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:28.337935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:28.341743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:28.342263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:28.342879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] PATH_INFO: `/v3/users/ad591b13df894fb39e192b0278d8f224` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:28.345303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad591b13df894fb39e192b0278d8f224)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:28.349539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:28.450608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad591b13df894fb39e192b0278d8f224'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:28.477467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad591b13df894fb39e192b0278d8f224'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:28.493605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad591b13df894fb39e192b0278d8f224'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:28.496140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad591b13df894fb39e192b0278d8f224'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:28.498136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad591b13df894fb39e192b0278d8f224'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:28.536653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1070/2125] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:28 2022] DELETE /identity/v3/users/ad591b13df894fb39e192b0278d8f224 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:28.547500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:28.548441 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:28.548606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:28.585940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:28.588447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:28.680464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1056/2126] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:28.691535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:28.693326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:28.785164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1542e-d53a-4f33-9c8b-90edd3e3b415 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:28.791072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:28.791873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:28.792299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:28.792647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:28.814870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:28.817399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:28.830152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1071/2127] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:29.485658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:29.487712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:29.502868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceecbe2-d9e9-4406-bf0b-7bed8633e9e0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:29.510761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:29.511746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:29.512045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:29.512337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] PATH_INFO: `/v3/projects/5d8da26390f44da1bf01aeca22d6a26b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:29.513992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8da26390f44da1bf01aeca22d6a26b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:29.516634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:29.518816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:29.638986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8da26390f44da1bf01aeca22d6a26b', 'request_id': 'req-8e554d62-e355-41ce-a047-1a7934df051e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.651021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8da26390f44da1bf01aeca22d6a26b', 'request_id': 'req-8e554d62-e355-41ce-a047-1a7934df051e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.661138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8da26390f44da1bf01aeca22d6a26b', 'request_id': 'req-8e554d62-e355-41ce-a047-1a7934df051e'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.683559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] The token cache is being invalidate because project 5d8da26390f44da1bf01aeca22d6a26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:29.684606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.686395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.691069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1057/2128] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:29 2022] DELETE /identity/v3/projects/5d8da26390f44da1bf01aeca22d6a26b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:29.697194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:29.715030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab639b0a-001d-4c37-9eb4-a7b96e94979f service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:29.808021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:29.830720 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:29.831620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:29.831908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:29.832186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] PATH_INFO: `/v3/users/cbfbb4c5c83a4a14ba83d8c086b8b1a8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:29.833476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfbb4c5c83a4a14ba83d8c086b8b1a8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:29.835519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:29.920912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfbb4c5c83a4a14ba83d8c086b8b1a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.930166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfbb4c5c83a4a14ba83d8c086b8b1a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.945206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfbb4c5c83a4a14ba83d8c086b8b1a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.946116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfbb4c5c83a4a14ba83d8c086b8b1a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.947044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-840915f4-ade4-4e95-835a-637dc5deddf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfbb4c5c83a4a14ba83d8c086b8b1a8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:29.973101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1072/2129] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:29 2022] DELETE /identity/v3/users/cbfbb4c5c83a4a14ba83d8c086b8b1a8 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:30.760615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:30.768742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:30.795831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e554d62-e355-41ce-a047-1a7934df051e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:30.919988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:30.921000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:30.921661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:30.922378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] PATH_INFO: `/v3/projects/63436d262b0c4871b7568b09940b544e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:30.924271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63436d262b0c4871b7568b09940b544e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:30.927373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.009323 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63436d262b0c4871b7568b09940b544e', 'request_id': 'req-65dc67d1-1df5-4280-877e-141f4a6eae7b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:31.018420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63436d262b0c4871b7568b09940b544e', 'request_id': 'req-65dc67d1-1df5-4280-877e-141f4a6eae7b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:31.031287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63436d262b0c4871b7568b09940b544e', 'request_id': 'req-65dc67d1-1df5-4280-877e-141f4a6eae7b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:31.087408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] The token cache is being invalidate because project 63436d262b0c4871b7568b09940b544e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:31.088955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:31.089838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-65dc67d1-1df5-4280-877e-141f4a6eae7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:31.096801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1058/2130] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:30 2022] DELETE /identity/v3/projects/63436d262b0c4871b7568b09940b544e => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:31.102864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.104889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.105778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.141549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:31.142960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:31.255562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1073/2131] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:31.264753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.265224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.265778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.296001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:31.297766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:31.309729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1059/2132] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:31.316238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:31.329066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54a52-32f4-43cd-9b21-70623ebc3a6c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:31.335536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:31.336471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.336804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.337132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.338129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:31.338625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:31.340517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.354661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1074/2133] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:31.359666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:31.370482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba2197-d232-4297-8daa-e96727f6396f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:31.379365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:31.380944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.381654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.382286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.383569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:31.387190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.409825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:31.449096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1060/2134] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:31 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:31.454010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:31.483131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477066b-374d-4a5c-a823-42da53b97cbc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:31.601100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:31.635284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:31.636985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.637547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.638265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.639650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:31.646871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.677865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1075/2135] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:31 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:31.682143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:31.700252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50edf35-a7d3-45bf-831c-e09b6477ced0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:31.709799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:31.711067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.711599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.712351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.714036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:31.717027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.741745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1061/2136] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:31.749672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:31.771551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030033d0-16c6-4706-bd9e-4adda2e44cd6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:31.780270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c1577-2a91-403c-809a-3423f58cb11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:31.781482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c1577-2a91-403c-809a-3423f58cb11c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.781840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c1577-2a91-403c-809a-3423f58cb11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.782202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c1577-2a91-403c-809a-3423f58cb11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.792172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c1577-2a91-403c-809a-3423f58cb11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:31.793097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c1577-2a91-403c-809a-3423f58cb11c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.807971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1076/2137] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:31.812671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:31.824282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41f94a-5cd8-4da8-aea5-b827bb9482a2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:31.830432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:31.831989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.832448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.832899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] PATH_INFO: `/v3/projects/04bec53a1bed4ec6a480e087169dfdc4/users/0fe49268277d458da7e2ac6cb41c0a8a/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.865142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bec53a1bed4ec6a480e087169dfdc4, user_id=0fe49268277d458da7e2ac6cb41c0a8a, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:31.868440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:31.894460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1062/2138] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:31 2022] PUT /identity/v3/projects/04bec53a1bed4ec6a480e087169dfdc4/users/0fe49268277d458da7e2ac6cb41c0a8a/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:31.908921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:31.909473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:31.909857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:31.952642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] MFA Rules not processed for user `0fe49268277d458da7e2ac6cb41c0a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:31.954855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:32.215081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1077/2139] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:32.220123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:32.415867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59b4f2-9a26-4ffa-89a8-a8ce94776596 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:32.421881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:32.422739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:32.423035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:32.423317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:32.424030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:32.426011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:32.508587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1063/2140] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:32 2022] POST /identity/v3/projects => generated 381 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:32.511168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:32.561708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6672405-eaf3-4566-9742-9e40499a1153 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:32.695455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:32.697770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:32.699184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:32.700051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:32.701854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:32.706234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:32.764293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1078/2141] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:32 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:32.770955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:32.785510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e3388-5d90-46f5-87ac-f6a5542ec155 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:32.801420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:32.802419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:32.802850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:32.803246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:32.804444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:32.808576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:32.838672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1064/2142] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:32.847249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:32.889852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf63c0f-8f6c-40b0-8a9f-1871da41002b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:32.904723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:32.907486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:32.907866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:32.908306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] PATH_INFO: `/v3/projects/63852738a2dc403b9daf45eb5d2f417c/users/710374ac80ff49038cc398aae040c617/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:32.939920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63852738a2dc403b9daf45eb5d2f417c, user_id=710374ac80ff49038cc398aae040c617, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:32.944751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:32.965730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1079/2143] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:32 2022] PUT /identity/v3/projects/63852738a2dc403b9daf45eb5d2f417c/users/710374ac80ff49038cc398aae040c617/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:32.970389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:32.983345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97f892-44de-4075-b21e-7c6d45de7fe9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:33.070037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:33.070876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.071168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.071452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.072390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:33.075391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:33.096808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1065/2144] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:33.102094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:33.115663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6a435-0320-43ac-a0ff-38a1d360d5d5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:33.125070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b89617-4260-4f32-b28f-54e31bf62e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:33.126628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89617-4260-4f32-b28f-54e31bf62e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.127135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89617-4260-4f32-b28f-54e31bf62e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.127626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b89617-4260-4f32-b28f-54e31bf62e8d admin admin] PATH_INFO: `/v3/projects/63852738a2dc403b9daf45eb5d2f417c/users/710374ac80ff49038cc398aae040c617/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.131364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89617-4260-4f32-b28f-54e31bf62e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63852738a2dc403b9daf45eb5d2f417c, user_id=710374ac80ff49038cc398aae040c617, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:33.134721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b89617-4260-4f32-b28f-54e31bf62e8d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:33.150149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1080/2145] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:33 2022] PUT /identity/v3/projects/63852738a2dc403b9daf45eb5d2f417c/users/710374ac80ff49038cc398aae040c617/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:33.165473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:33.167671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.171516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.171875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.172307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.194301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ea98e-5c7e-4eae-b3f7-930a655423de admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:33.207591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] MFA Rules not processed for user `710374ac80ff49038cc398aae040c617`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:33.208856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.215070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:33.230677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:33.268342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:33.269249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.269535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.269815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] PATH_INFO: `/v3/users/2a01407bfc0a480a8f6702993ddb0183` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.271442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a01407bfc0a480a8f6702993ddb0183)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:33.274745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:33.292101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1081/2146] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:33.304487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:33.305971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.350908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a01407bfc0a480a8f6702993ddb0183'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.366106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a01407bfc0a480a8f6702993ddb0183'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.367557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a01407bfc0a480a8f6702993ddb0183'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.368738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a01407bfc0a480a8f6702993ddb0183'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.381076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96b32e-11f8-455c-b553-e190b830d6b5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:33.382850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a01407bfc0a480a8f6702993ddb0183'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.385916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:33.386656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.387399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.387399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.389615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.414448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:33.417479 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:33.417673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1066/2147] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:33 2022] DELETE /identity/v3/users/2a01407bfc0a480a8f6702993ddb0183 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:33.433861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1082/2148] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:33.567224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:33.569629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.660140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67b7ee-fd23-4aee-ba3e-f3d8d2f78f67 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:33.671760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:33.672607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:33.673253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.673420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.673695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.673963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.676155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:33.702634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a7cc8-0d16-4471-a278-4c3e31cb57bc service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:33.787946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:33.789102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547719e0-d39d-46e0-bc49-3d7b6f25416b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:33.796733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:33.806351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1067/2149] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:33.838842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:33.840222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:33.840528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:33.840795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] PATH_INFO: `/v3/projects/f1097bdacbb44067bfa5dbf28bc86d9a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:33.842065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1097bdacbb44067bfa5dbf28bc86d9a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:33.844088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:33.924845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:33.928917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1097bdacbb44067bfa5dbf28bc86d9a', 'request_id': 'req-787f11d6-ed66-4eb0-8b08-a1741a329b53'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.957933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1097bdacbb44067bfa5dbf28bc86d9a', 'request_id': 'req-787f11d6-ed66-4eb0-8b08-a1741a329b53'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:33.996356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1097bdacbb44067bfa5dbf28bc86d9a', 'request_id': 'req-787f11d6-ed66-4eb0-8b08-a1741a329b53'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.022441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] The token cache is being invalidate because project f1097bdacbb44067bfa5dbf28bc86d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:34.023163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.025142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-787f11d6-ed66-4eb0-8b08-a1741a329b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.028093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1083/2150] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:33 2022] DELETE /identity/v3/projects/f1097bdacbb44067bfa5dbf28bc86d9a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:34.034670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.035348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.035348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.063595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:34.064783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.165443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1068/2151] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:34.180607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.180931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.181388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.204704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:34.206031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.220183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1084/2152] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:34.227007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.239713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae6f35-4b2c-46f4-a0da-dc7eb4616daf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.253175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.255061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.255633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.255980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.257335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:34.257876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.260919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.286187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.294616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1069/2153] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:34.303395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a288a-e2d3-4f3f-93b8-b7673de2bd24 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.310292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.311287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.312309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.312682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.313049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.313718 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.314055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.316566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.328491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ac608-914e-4afd-b307-aeb7ad49996e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.334718 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.335714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.336062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.336400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] PATH_INFO: `/v3/users/0fe49268277d458da7e2ac6cb41c0a8a` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.338072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fe49268277d458da7e2ac6cb41c0a8a)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.340669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.364257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1085/2154] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/projects => generated 363 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:34.369114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.415719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fe49268277d458da7e2ac6cb41c0a8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.440072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fe49268277d458da7e2ac6cb41c0a8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.440896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fe49268277d458da7e2ac6cb41c0a8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.441701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fe49268277d458da7e2ac6cb41c0a8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.456260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-501ea630-1dd4-4229-8d44-b0bea8d079f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fe49268277d458da7e2ac6cb41c0a8a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:34.482548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ab50d-0ea8-495e-b44c-8a84e59da4f2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.483122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1070/2155] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:34 2022] DELETE /identity/v3/users/0fe49268277d458da7e2ac6cb41c0a8a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:34.489244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.489540 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.490857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.511268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:34.512820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.576829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.577427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.577711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.577985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.578758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.580822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.593572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:34.610279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1071/2156] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:34.617204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1086/2157] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/users => generated 282 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:34.623531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.625256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.626273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.629185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.647478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040ba1-8bb1-4544-89a7-3f021977979f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.656178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.656992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.657421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.657773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.658722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.660526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.686865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1072/2158] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:34.689104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.702036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:34.706941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd071ad-fb82-4a51-8fb7-cd7ed9a28b5c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.712357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.713545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.713838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.714121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.714939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.716790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.721200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ba5a4-2d05-4353-bf42-cfbdedbe30a0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.726800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48ec36-61a7-40e2-bbd3-a982e44861ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.727655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ec36-61a7-40e2-bbd3-a982e44861ac service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.727975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ec36-61a7-40e2-bbd3-a982e44861ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.728305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ec36-61a7-40e2-bbd3-a982e44861ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.737324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1073/2159] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:34.739500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:34.744102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48ec36-61a7-40e2-bbd3-a982e44861ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.745953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48ec36-61a7-40e2-bbd3-a982e44861ac service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.755897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697572-cef6-479d-88e7-fc610b8a0cd8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:34.762042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1087/2160] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:34.762448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:34.763462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.763776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.764059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] PATH_INFO: `/v3/projects/de8f8105b075453fb7c9fffaa32099f0/users/a1706809dd844a4188e570f22a7cec1b/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.783115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8f8105b075453fb7c9fffaa32099f0, user_id=a1706809dd844a4188e570f22a7cec1b, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:34.786135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:34.811286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1074/2161] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:34 2022] PUT /identity/v3/projects/de8f8105b075453fb7c9fffaa32099f0/users/a1706809dd844a4188e570f22a7cec1b/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:34.826587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:34.826587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:34.826587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:34.861241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] MFA Rules not processed for user `a1706809dd844a4188e570f22a7cec1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:34.862568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:34.956415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1088/2162] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:34.962199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.074978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d58a9-e25c-4fd7-8c58-22de687e6f83 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.083809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.084537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.084826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.085137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.085886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.088521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.133985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1075/2163] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:34 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:35.139327 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.159735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f192018-992e-4c5e-b977-bcca905e5d76 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.197383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.198754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:35.280455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be1754-f519-4544-bd59-059463871946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.281827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be1754-f519-4544-bd59-059463871946 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.282347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be1754-f519-4544-bd59-059463871946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.282886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be1754-f519-4544-bd59-059463871946 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.284245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be1754-f519-4544-bd59-059463871946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.288448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be1754-f519-4544-bd59-059463871946 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.295608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcae144f-f53d-496a-82f8-a3ec62a4d18d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.304909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.307382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.308152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.308973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] PATH_INFO: `/v3/projects/04bec53a1bed4ec6a480e087169dfdc4` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.311960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bec53a1bed4ec6a480e087169dfdc4)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.314691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.329275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1089/2164] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:35 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:35.335371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be1754-f519-4544-bd59-059463871946 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.359589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be1754-f519-4544-bd59-059463871946 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.374827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.377268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.377779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.378317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.379968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.383559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.404090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1090/2165] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:35.404367 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:35.406421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.414328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bec53a1bed4ec6a480e087169dfdc4', 'request_id': 'req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.420781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49835b62-77c5-4f5b-bb16-50227bce97d7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.427937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bec53a1bed4ec6a480e087169dfdc4', 'request_id': 'req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.439316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bec53a1bed4ec6a480e087169dfdc4', 'request_id': 'req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.463789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] The token cache is being invalidate because project 04bec53a1bed4ec6a480e087169dfdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:35.464338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.465586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.468741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1076/2166] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:35 2022] DELETE /identity/v3/projects/04bec53a1bed4ec6a480e087169dfdc4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:35.475470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.478069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:35.510944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:35.527172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.528981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.529548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.530094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] PATH_INFO: `/v3/projects/01d2ca3905fd4eb2972fda59fc09b39d/users/9a63e0bf58294bbfa66ca027a5f861a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.555008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d2ca3905fd4eb2972fda59fc09b39d, user_id=9a63e0bf58294bbfa66ca027a5f861a8, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.558561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.579812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1091/2167] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:35 2022] PUT /identity/v3/projects/01d2ca3905fd4eb2972fda59fc09b39d/users/9a63e0bf58294bbfa66ca027a5f861a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:35.585881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.588905 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:35.599872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b7525-29d7-4c4b-b1c3-d92e7e4e0ed2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.695971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e422c7-44e1-42b8-985c-693752e3bf6c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.701177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.702192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.702483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.702687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.702785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] PATH_INFO: `/v3/users/710374ac80ff49038cc398aae040c617` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.703606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.704167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710374ac80ff49038cc398aae040c617)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.704333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.704444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.705240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.706244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.707043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.726579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1092/2168] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:35 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:35.731947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.746304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b377b-29c9-47e1-b137-37f386a46b4f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:35.754357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e92bf-be87-4494-bd86-eb5a3a76055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:35.755658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e92bf-be87-4494-bd86-eb5a3a76055f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.756279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e92bf-be87-4494-bd86-eb5a3a76055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.756642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e92bf-be87-4494-bd86-eb5a3a76055f admin admin] PATH_INFO: `/v3/projects/01d2ca3905fd4eb2972fda59fc09b39d/users/9a63e0bf58294bbfa66ca027a5f861a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.760994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e92bf-be87-4494-bd86-eb5a3a76055f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d2ca3905fd4eb2972fda59fc09b39d, user_id=9a63e0bf58294bbfa66ca027a5f861a8, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:35.767838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e92bf-be87-4494-bd86-eb5a3a76055f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:35.788205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710374ac80ff49038cc398aae040c617'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.789306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1093/2169] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:35 2022] PUT /identity/v3/projects/01d2ca3905fd4eb2972fda59fc09b39d/users/9a63e0bf58294bbfa66ca027a5f861a8/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:35.808930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710374ac80ff49038cc398aae040c617'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.810638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710374ac80ff49038cc398aae040c617'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.811691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710374ac80ff49038cc398aae040c617'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.811843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:35.812412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:35.813061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:35.828834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710374ac80ff49038cc398aae040c617'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:35.851828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] MFA Rules not processed for user `9a63e0bf58294bbfa66ca027a5f861a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:35.852747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:35.853193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:35.865675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1077/2170] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:35 2022] DELETE /identity/v3/users/710374ac80ff49038cc398aae040c617 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:35.966827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1094/2171] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:35.981105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:35.983572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:36.061796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49274c1-5316-419b-a207-e258739ad2f0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:36.065593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:36.066259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.066580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.066869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.180437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:36.183105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:36.200708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1078/2172] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:36.285886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:36.320058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:36.323376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:36.365568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:36.376726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22edba-1ced-41e4-866a-c5a371d29324 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:36.382415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:36.383508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.384059 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.384658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.386841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:36.430773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377194d-0673-4e95-813e-2eacecf05960 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:36.437799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:36.438970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.439340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.439715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] PATH_INFO: `/v3/projects/63852738a2dc403b9daf45eb5d2f417c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.441399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63852738a2dc403b9daf45eb5d2f417c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:36.444003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:36.499683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:36.501901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6eb09-62f7-415f-90df-8e13e3d05b6b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:36.516202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1095/2173] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:36.525882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:36.528862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63852738a2dc403b9daf45eb5d2f417c', 'request_id': 'req-6e6a390b-61c2-4c76-be9e-fb75394214c0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:36.541394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63852738a2dc403b9daf45eb5d2f417c', 'request_id': 'req-6e6a390b-61c2-4c76-be9e-fb75394214c0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:36.552316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63852738a2dc403b9daf45eb5d2f417c', 'request_id': 'req-6e6a390b-61c2-4c76-be9e-fb75394214c0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:36.581530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] The token cache is being invalidate because project 63852738a2dc403b9daf45eb5d2f417c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:36.582209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:36.584292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e6a390b-61c2-4c76-be9e-fb75394214c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:36.587157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1079/2174] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:36 2022] DELETE /identity/v3/projects/63852738a2dc403b9daf45eb5d2f417c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:36.593275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.593574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.593867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.624214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:36.626398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:36.742557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.742557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.742557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.747208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1096/2175] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:36.766141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:36.767728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:36.778583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1080/2176] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:36.782473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:36.800320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd27d9-26de-40e6-8d27-069c7db2ed03 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:36.807248 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:36.808346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.808832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.809369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.810747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:36.811271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:36.814336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:36.834953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1097/2177] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:36.842077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:36.852415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb00fc-bde0-413a-b111-db1b300293ce None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:36.859701 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:36.860678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:36.861081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:36.861368 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:36.862501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:36.866085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:36.910489 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1081/2178] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:36 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:36.916929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:36.931567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcc97a-3472-43a4-9322-7c943b5f9054 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.029617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.031323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.032206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.033144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.035777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.039007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.063953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1098/2179] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:36 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:37.068553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.080381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c01096-66cd-47a7-8543-979a6dcf598e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.085544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.086234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.086542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.086990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.088011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.089454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.102442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1082/2180] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:37.109494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.128211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc35e8f-a00c-4b4a-b49d-cd8993bba69d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.136495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71280c9e-500e-43fa-8013-48bcecfbb0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.137664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71280c9e-500e-43fa-8013-48bcecfbb0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.138166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71280c9e-500e-43fa-8013-48bcecfbb0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.138691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71280c9e-500e-43fa-8013-48bcecfbb0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.140095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71280c9e-500e-43fa-8013-48bcecfbb0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.143354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71280c9e-500e-43fa-8013-48bcecfbb0b9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.156083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1099/2181] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:37.160887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.172142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac0d5e-e94c-4463-a3c0-077b0b5a3081 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.177401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.178408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.178726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.179041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] PATH_INFO: `/v3/projects/f90328626a7f4258ac8b5f78189ff4f5/users/30ca27cf5f654d36bcefd8d8e405d6d2/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.195836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90328626a7f4258ac8b5f78189ff4f5, user_id=30ca27cf5f654d36bcefd8d8e405d6d2, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.199095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.217459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1083/2182] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:37 2022] PUT /identity/v3/projects/f90328626a7f4258ac8b5f78189ff4f5/users/30ca27cf5f654d36bcefd8d8e405d6d2/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:37.233286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.233833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.234501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.257752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:37.283837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] MFA Rules not processed for user `30ca27cf5f654d36bcefd8d8e405d6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:37.285722 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:37.372482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1100/2183] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:37.377749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.484060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a3288-92fc-478d-9949-6c0be3fd12f8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.492463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.493171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.493456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.493736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.494497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.496593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.532870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1084/2184] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:37 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:37.537900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.555047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455d2f8-ff6e-416e-8086-fa486d26fe88 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.630665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.631356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.631919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.632197 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.632895 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.635309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.662923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1101/2185] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:37 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:37.671643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.692780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3002c-2cee-43c9-8517-3efeb45d7c9a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.711759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.713405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.713795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.714196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.715627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.718779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.740633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1085/2186] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:37.745782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.759638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca590a79-65cc-4103-9c49-0a8a419a7b74 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.767369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c99977-e515-4219-a11e-a5c1d3f110f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.767916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99977-e515-4219-a11e-a5c1d3f110f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.768356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99977-e515-4219-a11e-a5c1d3f110f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.768826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c99977-e515-4219-a11e-a5c1d3f110f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.769921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c99977-e515-4219-a11e-a5c1d3f110f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.774666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c99977-e515-4219-a11e-a5c1d3f110f4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.798791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1102/2187] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:37.801456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:37.818823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46936064-93e4-453f-8a00-3cd922813cb6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:37.827572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:37.834509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.835038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.835556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] PATH_INFO: `/v3/projects/5651672e3aea4edb86d4ac63757d1025/users/9f999f13a83e477b94e810e0cd6690f9/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.866334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5651672e3aea4edb86d4ac63757d1025, user_id=9f999f13a83e477b94e810e0cd6690f9, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:37.872242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:37.900560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1086/2188] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:37 2022] PUT /identity/v3/projects/5651672e3aea4edb86d4ac63757d1025/users/9f999f13a83e477b94e810e0cd6690f9/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:37.918660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:37.919027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:37.919375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:37.970055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] MFA Rules not processed for user `9f999f13a83e477b94e810e0cd6690f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:37.972342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:38.074727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1103/2189] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:38.083138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.117855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:38.173816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dce275-6166-4a24-abe0-b61fb5e16cba admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.191787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.192496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.192760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.193037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.193801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.195729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:38.233111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1087/2190] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:38 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:38.234577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.242067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:38.248343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74eeaf9-1434-4f68-997d-b359fc441459 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.349974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.350834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.351185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.351522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.352478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.354937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:38.381182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1104/2191] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:38 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:38.386381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.398282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bff7d-dfd0-4f8a-9f06-2735adb06adf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.404327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.405110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.405444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.405801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.406668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.408423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:38.422308 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1088/2192] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:38.427388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.441909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16b37d-a6d0-46d9-ba04-61e742e731da admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.448798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.449939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.450297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.450695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] PATH_INFO: `/v3/projects/b5d46885820d4009aa6540a9f18a32a5/users/b2b3f68c0bf34b3b87fa66540aa60dba/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.468855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d46885820d4009aa6540a9f18a32a5, user_id=b2b3f68c0bf34b3b87fa66540aa60dba, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.471821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:38.494566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1105/2193] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:38 2022] PUT /identity/v3/projects/b5d46885820d4009aa6540a9f18a32a5/users/b2b3f68c0bf34b3b87fa66540aa60dba/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:38.498864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.510585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c90c4b-28af-439d-bd08-3244e3d35b83 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.571856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60df99-4180-4a51-adec-6bcfbe27d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.572520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60df99-4180-4a51-adec-6bcfbe27d3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.572900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60df99-4180-4a51-adec-6bcfbe27d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.573273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60df99-4180-4a51-adec-6bcfbe27d3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.574142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60df99-4180-4a51-adec-6bcfbe27d3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.576075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60df99-4180-4a51-adec-6bcfbe27d3f7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:38.587553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1089/2194] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:38.592574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.603764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c42210-b9aa-4e14-be97-101ff7840109 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.608779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.609826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.610151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.610472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] PATH_INFO: `/v3/projects/b5d46885820d4009aa6540a9f18a32a5/users/b2b3f68c0bf34b3b87fa66540aa60dba/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.612654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d46885820d4009aa6540a9f18a32a5, user_id=b2b3f68c0bf34b3b87fa66540aa60dba, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.615446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:38.627125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1106/2195] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:38 2022] PUT /identity/v3/projects/b5d46885820d4009aa6540a9f18a32a5/users/b2b3f68c0bf34b3b87fa66540aa60dba/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:38.639189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.639479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.639753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.673812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] MFA Rules not processed for user `b2b3f68c0bf34b3b87fa66540aa60dba`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:38.675108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:38.768301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1090/2196] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:38.786790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:38.789483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:38.867240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c34e8-dda1-4bcf-8a34-b3e540096c8c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:38.874246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422ebf0-9119-468c-a5a6-67a87787694a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:38.875425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422ebf0-9119-468c-a5a6-67a87787694a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:38.875921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422ebf0-9119-468c-a5a6-67a87787694a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:38.876372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422ebf0-9119-468c-a5a6-67a87787694a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:38.994299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422ebf0-9119-468c-a5a6-67a87787694a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:38.996552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422ebf0-9119-468c-a5a6-67a87787694a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:39.011326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1107/2197] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:39.602043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:39.603737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:39.629886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:39.709355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f4f1c-ae36-442e-ab20-929608c2d7e9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:39.717865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:39.719460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:39.719952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:39.720461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] PATH_INFO: `/v3/users/a1706809dd844a4188e570f22a7cec1b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:39.722841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1706809dd844a4188e570f22a7cec1b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:39.726683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:39.787720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1706809dd844a4188e570f22a7cec1b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:39.800416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1706809dd844a4188e570f22a7cec1b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:39.801451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1706809dd844a4188e570f22a7cec1b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:39.802377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1706809dd844a4188e570f22a7cec1b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:39.811802 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1706809dd844a4188e570f22a7cec1b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:39.833312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1091/2198] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:39 2022] DELETE /identity/v3/users/a1706809dd844a4188e570f22a7cec1b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:39.838794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:39.839406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:39.839406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:39.859057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:39.860226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:39.866111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:39.980906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1108/2199] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:39.991652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:39.994832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:40.076983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897943c4-017e-4712-8e8b-1aa66bd9bf57 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:40.083043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:40.084175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:40.084725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:40.084975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:40.103855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:40.107154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:40.136513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1092/2200] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:40.628951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:40.631654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:40.658495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992bd82-4cf4-4b39-ab5a-aeb719f5a87d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:40.665785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:40.666855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:40.667461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:40.667820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] PATH_INFO: `/v3/projects/de8f8105b075453fb7c9fffaa32099f0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:40.670015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8f8105b075453fb7c9fffaa32099f0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:40.672860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:40.727364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8f8105b075453fb7c9fffaa32099f0', 'request_id': 'req-a400c625-1e84-4c63-9c0a-faa5f45ee706'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.737458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8f8105b075453fb7c9fffaa32099f0', 'request_id': 'req-a400c625-1e84-4c63-9c0a-faa5f45ee706'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.745848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8f8105b075453fb7c9fffaa32099f0', 'request_id': 'req-a400c625-1e84-4c63-9c0a-faa5f45ee706'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.766644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] The token cache is being invalidate because project de8f8105b075453fb7c9fffaa32099f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:40.767157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.768678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.771166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1109/2201] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:40 2022] DELETE /identity/v3/projects/de8f8105b075453fb7c9fffaa32099f0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:40.775488 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:40.790495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8e955-586a-41f9-b0cf-420871b29582 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:40.850346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:40.877313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:40.878161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:40.878462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:40.878784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] PATH_INFO: `/v3/users/9a63e0bf58294bbfa66ca027a5f861a8` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:40.881798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a63e0bf58294bbfa66ca027a5f861a8)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:40.883868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:40.956873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a63e0bf58294bbfa66ca027a5f861a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.975641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a63e0bf58294bbfa66ca027a5f861a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.976410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a63e0bf58294bbfa66ca027a5f861a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.977170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a63e0bf58294bbfa66ca027a5f861a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:40.986776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a63e0bf58294bbfa66ca027a5f861a8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:41.011495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1093/2202] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:40 2022] DELETE /identity/v3/users/9a63e0bf58294bbfa66ca027a5f861a8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:41.791650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:41.795345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:41.827090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c625-1e84-4c63-9c0a-faa5f45ee706 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:41.968551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:41.971440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:41.971884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:41.972288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] PATH_INFO: `/v3/projects/01d2ca3905fd4eb2972fda59fc09b39d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:41.976563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d2ca3905fd4eb2972fda59fc09b39d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:41.984872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.043927 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.046911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:42.060237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:42.076957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d2ca3905fd4eb2972fda59fc09b39d', 'request_id': 'req-d39fbbae-a4a1-443b-99db-79b9b60d0441'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:42.090210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d2ca3905fd4eb2972fda59fc09b39d', 'request_id': 'req-d39fbbae-a4a1-443b-99db-79b9b60d0441'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:42.100023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d2ca3905fd4eb2972fda59fc09b39d', 'request_id': 'req-d39fbbae-a4a1-443b-99db-79b9b60d0441'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:42.131220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] The token cache is being invalidate because project 01d2ca3905fd4eb2972fda59fc09b39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:42.131606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:42.136911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d39fbbae-a4a1-443b-99db-79b9b60d0441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:42.144439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1110/2203] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:41 2022] DELETE /identity/v3/projects/01d2ca3905fd4eb2972fda59fc09b39d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:42.146318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61560691-96f9-4aa6-bbe0-3bda4a89d4a7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.146769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61560691-96f9-4aa6-bbe0-3bda4a89d4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.147251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61560691-96f9-4aa6-bbe0-3bda4a89d4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.178344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-61560691-96f9-4aa6-bbe0-3bda4a89d4a7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:42.179820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddc45e-603d-42f5-99ea-22c51c22d530 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.180721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-61560691-96f9-4aa6-bbe0-3bda4a89d4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:42.262174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.263429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.263983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.264546 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.268696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:42.279917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1111/2204] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:42.288065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.288358 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.288634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.308855 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:42.310095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:42.323946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1112/2205] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:42.324403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:42.329055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.347554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c1f54f-39be-4c44-83df-79a2afcb618e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.368788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.370195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.370934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.371784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.373957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:42.374561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.378071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.382464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.385504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.396130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1094/2206] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:42.408847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1113/2207] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:28:42.413827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.428075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ea091-54a9-44b0-9b44-52762edffc1b service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.433678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.434364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.434693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.434977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.435713 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.437686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.487615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1095/2208] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:42 2022] POST /identity/v3/projects => generated 399 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:42.491860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.512656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c7fe5-973e-4b0a-9ed0-78b3e8095792 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.609514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.611052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.612497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.614192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.615844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.619774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.658982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1114/2209] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:42 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:42.664135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.676876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cee805-97e4-42ec-9a7e-aab146eaf0f4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.692590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.693286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.693570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.693849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.694629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.696272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.727020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1096/2210] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:42.732711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.743900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e51a9-26e2-4857-b229-66b7cb6e257b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.749909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f005e-e5a0-4d71-b14d-ef5f9d2b35d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.751304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f005e-e5a0-4d71-b14d-ef5f9d2b35d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.751569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f005e-e5a0-4d71-b14d-ef5f9d2b35d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.751843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f005e-e5a0-4d71-b14d-ef5f9d2b35d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.752703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f005e-e5a0-4d71-b14d-ef5f9d2b35d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.754648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f005e-e5a0-4d71-b14d-ef5f9d2b35d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.779377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1115/2211] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:42.781201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:42.802986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb73a5-b98e-4050-8ca5-2ab72e07518f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:42.823927 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:42.826250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.826740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.827045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] PATH_INFO: `/v3/projects/814ea66f601d41ac941ff040a5f548a6/users/a586ad20932144ba8ed59044a4c66499/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.857245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814ea66f601d41ac941ff040a5f548a6, user_id=a586ad20932144ba8ed59044a4c66499, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:42.863462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:42.893431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1097/2212] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:42 2022] PUT /identity/v3/projects/814ea66f601d41ac941ff040a5f548a6/users/a586ad20932144ba8ed59044a4c66499/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:42.913622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:42.914734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:42.915330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:42.969639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] MFA Rules not processed for user `a586ad20932144ba8ed59044a4c66499`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:42.971310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:43.017564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:43.177057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1116/2213] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:43.186124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.318750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67febfa4-b7f9-465f-a4ee-74c64abd7b11 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.328299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.329521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.329906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.330191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.331065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.334446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.357840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.360042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:43.378472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1098/2214] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:43 2022] POST /identity/v3/projects => generated 401 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:43.384768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.461582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bc3d8-e208-4cb0-9a1f-7c8067e625ac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.469964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:43.475901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346fb04-76d4-44f9-9d7f-7f2248d38f90 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.483261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.485152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.485710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.486264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.487464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.491305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.527094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1099/2215] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:43 2022] POST /identity/v3/users => generated 307 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:43.532630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.539847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.543209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.543526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.543843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.553770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:43.556563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd6144-dd99-43a7-b45e-bd4a9e10aa40 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.563898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.564660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.564974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.565391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.566384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.568089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.581544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1100/2216] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:43.586377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.597728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6308-4058-4cff-839d-bc54880d10e7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.603866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.604830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.605119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.605399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] PATH_INFO: `/v3/projects/3001b98cc8224cd4b53330ca7e3be522/users/edd22c35b59242e9bb2dad6778a7f5a7/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.629342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3001b98cc8224cd4b53330ca7e3be522, user_id=edd22c35b59242e9bb2dad6778a7f5a7, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.632961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.652913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1101/2217] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:43 2022] PUT /identity/v3/projects/3001b98cc8224cd4b53330ca7e3be522/users/edd22c35b59242e9bb2dad6778a7f5a7/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:43.656914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.657608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.659269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.675646 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1117/2218] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:43.760906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abc5c5-8980-4997-8fb9-4e0676d85631 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.782291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d165f88-9bc6-4652-8e2d-6d1203a64d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.783640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d165f88-9bc6-4652-8e2d-6d1203a64d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.784386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d165f88-9bc6-4652-8e2d-6d1203a64d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.784942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d165f88-9bc6-4652-8e2d-6d1203a64d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.786456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d165f88-9bc6-4652-8e2d-6d1203a64d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.789845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d165f88-9bc6-4652-8e2d-6d1203a64d91 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.812574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1102/2219] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:43.814667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:43.832352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771df9a-17cf-4f63-8553-3e2a6c335ebc service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:43.839931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:43.841253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.841583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.842013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] PATH_INFO: `/v3/projects/3001b98cc8224cd4b53330ca7e3be522/users/edd22c35b59242e9bb2dad6778a7f5a7/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.845547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3001b98cc8224cd4b53330ca7e3be522, user_id=edd22c35b59242e9bb2dad6778a7f5a7, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:43.850573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:43.863860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1118/2220] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:43 2022] PUT /identity/v3/projects/3001b98cc8224cd4b53330ca7e3be522/users/edd22c35b59242e9bb2dad6778a7f5a7/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:43.876202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:43.876478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:43.876746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:43.910873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] MFA Rules not processed for user `edd22c35b59242e9bb2dad6778a7f5a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:43.912572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:44.033505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1103/2221] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:44.041759 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:44.057650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:44.146713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609946bb-1f79-4b86-9361-b2859d6b05d8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:44.150885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:44.151517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:44.151794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:44.152072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:44.163034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:44.164525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:44.176406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1119/2222] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:44.316233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:44.343279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0081299-513b-4b1d-9266-66d35d96af19 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:44.355703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a9ad7-51d0-4e6f-9d63-1b64866b5604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:44.356471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a9ad7-51d0-4e6f-9d63-1b64866b5604 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:44.356821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a9ad7-51d0-4e6f-9d63-1b64866b5604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:44.357282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a9ad7-51d0-4e6f-9d63-1b64866b5604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:44.412351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:44.462407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a9ad7-51d0-4e6f-9d63-1b64866b5604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:44.465212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a9ad7-51d0-4e6f-9d63-1b64866b5604 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:44.482643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1104/2223] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:44.494324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:44.495968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:44.585645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5973c2-c31b-4f0d-ab5e-fdabce09241b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:44.591626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:44.592525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:44.592811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:44.593089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] PATH_INFO: `/v3/users/7b0ddc2bf6d14f6c812ebe842b65c98c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:44.594396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0ddc2bf6d14f6c812ebe842b65c98c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:44.596517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:44.661787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0ddc2bf6d14f6c812ebe842b65c98c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:44.669963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0ddc2bf6d14f6c812ebe842b65c98c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:44.680633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0ddc2bf6d14f6c812ebe842b65c98c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:44.681474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0ddc2bf6d14f6c812ebe842b65c98c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:44.682276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0ddc2bf6d14f6c812ebe842b65c98c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:44.714770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1120/2224] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:44 2022] DELETE /identity/v3/users/7b0ddc2bf6d14f6c812ebe842b65c98c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:44.720303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:44.720820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:44.722002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:44.756033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:44.758109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:44.891218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1105/2225] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:44.909141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:44.912370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:44.993432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:45.043911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e433e5a-9a64-43af-b27f-f733f09350b3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:45.049496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2b3a2-11cd-42ee-8620-cb306b48fdc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:45.050090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2b3a2-11cd-42ee-8620-cb306b48fdc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:45.050417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2b3a2-11cd-42ee-8620-cb306b48fdc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:45.050751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2b3a2-11cd-42ee-8620-cb306b48fdc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:45.061821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2b3a2-11cd-42ee-8620-cb306b48fdc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:45.063339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2b3a2-11cd-42ee-8620-cb306b48fdc7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:45.074930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1121/2226] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:45.454086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:45.457149 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:45.474897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4535a0-9d2c-4a94-ab5f-9a1f19323ad0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:45.483646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:45.485331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:45.485915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:45.486501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] PATH_INFO: `/v3/projects/6ccfde95485141febad498eb7f8df7a3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:45.489128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ccfde95485141febad498eb7f8df7a3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:45.495683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:45.576938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ccfde95485141febad498eb7f8df7a3', 'request_id': 'req-0a0d6fec-9158-4841-9994-63244db29a7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:45.589299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ccfde95485141febad498eb7f8df7a3', 'request_id': 'req-0a0d6fec-9158-4841-9994-63244db29a7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:45.600081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ccfde95485141febad498eb7f8df7a3', 'request_id': 'req-0a0d6fec-9158-4841-9994-63244db29a7d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:45.623683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] The token cache is being invalidate because project 6ccfde95485141febad498eb7f8df7a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:45.625255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:45.626011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-0a0d6fec-9158-4841-9994-63244db29a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:45.627750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1106/2227] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:45 2022] DELETE /identity/v3/projects/6ccfde95485141febad498eb7f8df7a3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:45.636225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:45.636743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:45.637119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:45.656091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:45.657427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:45.744601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1122/2228] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:45.752706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:45.753287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:45.753465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:45.776605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:45.777865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:45.786553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1107/2229] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:45.792115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:45.808941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438d41a-8238-4e89-8dab-010a6ee7bc07 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:45.817469 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:45.818673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:45.819160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:45.819634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:45.820988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:45.822244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:45.825741 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:45.852504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1123/2230] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:45.853834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:45.869459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fb489-5a0c-41ec-b0fc-f12716a338d9 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:45.877242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:45.878387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:45.878927 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:45.879426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:45.880656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:45.884149 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:45.939567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1108/2231] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:45 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:45.941889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:45.956679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a0a4-ff98-42a1-8056-95ce79477a4a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:46.049816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:46.050729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:46.051236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:46.051657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:46.052477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:46.055405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:46.084038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1124/2232] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:45 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:46.088310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:46.099127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b3aa5-327a-4f8c-91ce-340c7ad53386 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:46.104046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:46.104726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:46.105011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:46.105288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:46.106031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:46.107710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:46.119474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1109/2233] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:46.123707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:46.137507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc447212-7035-4236-af29-246450e91b68 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:46.143149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09c0b5-b673-4fa1-9b02-036a56408d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:46.143842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09c0b5-b673-4fa1-9b02-036a56408d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:46.144146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09c0b5-b673-4fa1-9b02-036a56408d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:46.144407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09c0b5-b673-4fa1-9b02-036a56408d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:46.145156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09c0b5-b673-4fa1-9b02-036a56408d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:46.146838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09c0b5-b673-4fa1-9b02-036a56408d75 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:46.164023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1125/2234] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:46.170057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:46.189468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4032a-48fc-4ab1-bc91-591354965099 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:46.197145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:46.198764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:46.199239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:46.199655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] PATH_INFO: `/v3/projects/1c2735bba83f4ea19182c12c13b65601/users/c2101f9a4f394bd79239cad2f0b83ea6/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:46.218271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:46.224270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2735bba83f4ea19182c12c13b65601, user_id=c2101f9a4f394bd79239cad2f0b83ea6, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:46.228464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:46.252317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1110/2235] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:46 2022] PUT /identity/v3/projects/1c2735bba83f4ea19182c12c13b65601/users/c2101f9a4f394bd79239cad2f0b83ea6/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:46.265332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:46.266440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:46.266960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:46.306399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] MFA Rules not processed for user `c2101f9a4f394bd79239cad2f0b83ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:46.308493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:46.357668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:46.421017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1126/2236] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:46.441147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:46.443482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:46.510745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dee3e-de4b-416b-b3cd-7a83592b8ff8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:46.515105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff383fcc-2185-480a-be9b-ae501e18509f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:46.515806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff383fcc-2185-480a-be9b-ae501e18509f service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:46.516088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff383fcc-2185-480a-be9b-ae501e18509f service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:46.516361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff383fcc-2185-480a-be9b-ae501e18509f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:46.527452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff383fcc-2185-480a-be9b-ae501e18509f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:46.529030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff383fcc-2185-480a-be9b-ae501e18509f service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:46.540030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1111/2237] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:49.032635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:49.034505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:49.164061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ce4bf-b50d-4cbf-af04-cfcf1af3bfb4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:49.195626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:49.197318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:49.197881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:49.199025 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] PATH_INFO: `/v3/users/30ca27cf5f654d36bcefd8d8e405d6d2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:49.201272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ca27cf5f654d36bcefd8d8e405d6d2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:49.205525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:49.287567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ca27cf5f654d36bcefd8d8e405d6d2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:49.299634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ca27cf5f654d36bcefd8d8e405d6d2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:49.324487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ca27cf5f654d36bcefd8d8e405d6d2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:49.325444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ca27cf5f654d36bcefd8d8e405d6d2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:49.326576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ca27cf5f654d36bcefd8d8e405d6d2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:49.374337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1127/2238] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:49 2022] DELETE /identity/v3/users/30ca27cf5f654d36bcefd8d8e405d6d2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:49.380184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:49.381020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:49.381020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:49.416891 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:49.420141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:49.554298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1112/2239] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:49.573106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:49.575073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:49.671185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:49.691152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3f637-5405-4fe5-963b-d719a23a31c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:49.696750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:49.697425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:49.697717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:49.697980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:49.714257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:49.716714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:49.731793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1128/2240] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:50.070000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:50.072126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:50.084994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51939b1c-03e3-4737-8245-14c0b152b406 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:50.090486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:50.091415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:50.091689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:50.091962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] PATH_INFO: `/v3/users/a586ad20932144ba8ed59044a4c66499` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:50.093339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a586ad20932144ba8ed59044a4c66499)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:50.095479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:50.160531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:50.164075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:50.193776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b209e229-5da1-471f-82d0-a6dea4cad319 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:50.203644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a586ad20932144ba8ed59044a4c66499'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.203910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:50.205915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:50.206638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:50.207285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] PATH_INFO: `/v3/projects/f90328626a7f4258ac8b5f78189ff4f5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:50.209920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90328626a7f4258ac8b5f78189ff4f5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:50.215308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:50.232112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a586ad20932144ba8ed59044a4c66499'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.234404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a586ad20932144ba8ed59044a4c66499'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.235468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a586ad20932144ba8ed59044a4c66499'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.246851 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1d853af8-ef61-49e3-ba98-544720021e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a586ad20932144ba8ed59044a4c66499'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.291498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1113/2241] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:50 2022] DELETE /identity/v3/users/a586ad20932144ba8ed59044a4c66499 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:50.298594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e866-c263-4017-a086-69f680f59848 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:50.298890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e866-c263-4017-a086-69f680f59848 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:50.299373 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6e866-c263-4017-a086-69f680f59848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:50.318407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:50.321522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90328626a7f4258ac8b5f78189ff4f5', 'request_id': 'req-01b916b6-afc7-45aa-8264-0b36f65e56f8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.336537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-60c6e866-c263-4017-a086-69f680f59848 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:50.339709 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90328626a7f4258ac8b5f78189ff4f5', 'request_id': 'req-01b916b6-afc7-45aa-8264-0b36f65e56f8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.342478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-60c6e866-c263-4017-a086-69f680f59848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:50.353505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90328626a7f4258ac8b5f78189ff4f5', 'request_id': 'req-01b916b6-afc7-45aa-8264-0b36f65e56f8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.369848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] The token cache is being invalidate because project f90328626a7f4258ac8b5f78189ff4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:50.370171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.370888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.373365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1129/2242] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:50 2022] DELETE /identity/v3/projects/f90328626a7f4258ac8b5f78189ff4f5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:50.378398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:50.380656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:50.449682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1114/2243] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:50.463170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6e866-c263-4017-a086-69f680f59848 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:50.464996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-60c6e866-c263-4017-a086-69f680f59848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:50.483121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b916b6-afc7-45aa-8264-0b36f65e56f8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:50.491714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:50.492674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:50.493033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:50.493324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] PATH_INFO: `/v3/users/9f999f13a83e477b94e810e0cd6690f9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:50.494772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f999f13a83e477b94e810e0cd6690f9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:50.496740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:50.544360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6e866-c263-4017-a086-69f680f59848 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:50.548465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:50.549145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:50.549477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:50.549954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:50.562266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:50.564847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:50.574739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f999f13a83e477b94e810e0cd6690f9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.582350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1115/2244] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:50.587729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f999f13a83e477b94e810e0cd6690f9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.600164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f999f13a83e477b94e810e0cd6690f9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.601162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f999f13a83e477b94e810e0cd6690f9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.602477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f999f13a83e477b94e810e0cd6690f9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:50.616253 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:50.632573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1130/2245] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:50 2022] DELETE /identity/v3/users/9f999f13a83e477b94e810e0cd6690f9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:51.076260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:51.078147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:51.104902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae819954-501e-4ba1-bc61-124c092b3093 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:51.111331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:51.113384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:51.209336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:51.210224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:51.210501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:51.210838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] PATH_INFO: `/v3/projects/814ea66f601d41ac941ff040a5f548a6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:51.212239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=814ea66f601d41ac941ff040a5f548a6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:51.214288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:51.222312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3994-9307-4e78-bcf9-5f83319a36c4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:51.229271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:51.230341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:51.230717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:51.231066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] PATH_INFO: `/v3/projects/5651672e3aea4edb86d4ac63757d1025` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:51.232740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5651672e3aea4edb86d4ac63757d1025)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:51.235318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:51.276409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814ea66f601d41ac941ff040a5f548a6', 'request_id': 'req-e54c2f68-4b71-44be-8099-fb78bc48dd5d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.279362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:51.287407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814ea66f601d41ac941ff040a5f548a6', 'request_id': 'req-e54c2f68-4b71-44be-8099-fb78bc48dd5d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.300000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814ea66f601d41ac941ff040a5f548a6', 'request_id': 'req-e54c2f68-4b71-44be-8099-fb78bc48dd5d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.310865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5651672e3aea4edb86d4ac63757d1025', 'request_id': 'req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.321476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5651672e3aea4edb86d4ac63757d1025', 'request_id': 'req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.323232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] The token cache is being invalidate because project 814ea66f601d41ac941ff040a5f548a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:51.323440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.324146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.326122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1116/2246] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:51 2022] DELETE /identity/v3/projects/814ea66f601d41ac941ff040a5f548a6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:51.329971 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5651672e3aea4edb86d4ac63757d1025', 'request_id': 'req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.330205 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:51.332086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:51.346618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] The token cache is being invalidate because project 5651672e3aea4edb86d4ac63757d1025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:51.347274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.351417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.357518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1131/2247] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:51 2022] DELETE /identity/v3/projects/5651672e3aea4edb86d4ac63757d1025 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:51.360767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:51.363115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:51.426741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c2f68-4b71-44be-8099-fb78bc48dd5d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:51.473202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa965bbf-19a1-48ac-bdd6-39c250d62f9b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:51.480348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:51.481265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:51.481614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:51.481906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] PATH_INFO: `/v3/users/b2b3f68c0bf34b3b87fa66540aa60dba` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:51.484097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2b3f68c0bf34b3b87fa66540aa60dba)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:51.486224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:51.528632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:51.529393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:51.529799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:51.530290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] PATH_INFO: `/v3/users/edd22c35b59242e9bb2dad6778a7f5a7` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:51.534968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd22c35b59242e9bb2dad6778a7f5a7)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:51.538679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:51.544684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:51.587729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2b3f68c0bf34b3b87fa66540aa60dba'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.605399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2b3f68c0bf34b3b87fa66540aa60dba'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.616321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2b3f68c0bf34b3b87fa66540aa60dba'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.617010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2b3f68c0bf34b3b87fa66540aa60dba'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.617880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2b3f68c0bf34b3b87fa66540aa60dba'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.632726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd22c35b59242e9bb2dad6778a7f5a7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.646505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd22c35b59242e9bb2dad6778a7f5a7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.647383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd22c35b59242e9bb2dad6778a7f5a7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.648380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd22c35b59242e9bb2dad6778a7f5a7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.655705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1132/2248] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:51 2022] DELETE /identity/v3/users/b2b3f68c0bf34b3b87fa66540aa60dba => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:51.668057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd22c35b59242e9bb2dad6778a7f5a7'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:51.704202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1117/2249] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:51 2022] DELETE /identity/v3/users/edd22c35b59242e9bb2dad6778a7f5a7 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:52.148883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:52.151531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:52.251960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:52.253625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:52.267795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:52.339328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe363e-a5b3-4de8-acc4-23ee8d5c51b0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:52.401453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee62001-7404-4bc2-a9ff-9844f1a9ae4f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:52.411103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:52.413021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.413789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.414241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] PATH_INFO: `/v3/projects/3001b98cc8224cd4b53330ca7e3be522` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.421551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3001b98cc8224cd4b53330ca7e3be522)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:52.433199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:52.506398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3001b98cc8224cd4b53330ca7e3be522', 'request_id': 'req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.513760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:52.514043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3001b98cc8224cd4b53330ca7e3be522', 'request_id': 'req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.515592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.516092 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.516586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] PATH_INFO: `/v3/projects/b5d46885820d4009aa6540a9f18a32a5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.519593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d46885820d4009aa6540a9f18a32a5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:52.522220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3001b98cc8224cd4b53330ca7e3be522', 'request_id': 'req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.522886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:52.549199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] The token cache is being invalidate because project 3001b98cc8224cd4b53330ca7e3be522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:52.549199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.549199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e090fba6-d1f9-4cf7-bb95-fda57bb2dd28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.554577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1118/2250] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:52 2022] DELETE /identity/v3/projects/3001b98cc8224cd4b53330ca7e3be522 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:52.560288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41640-bf6e-4727-9e18-8d00a6f44878 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.560651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41640-bf6e-4727-9e18-8d00a6f44878 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.561378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd41640-bf6e-4727-9e18-8d00a6f44878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.595945 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d46885820d4009aa6540a9f18a32a5', 'request_id': 'req-3f4e846d-a6d1-428e-8027-f3bfb2cde015'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.596235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-5cd41640-bf6e-4727-9e18-8d00a6f44878 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:52.598286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:52.599102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5cd41640-bf6e-4727-9e18-8d00a6f44878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:52.609819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d46885820d4009aa6540a9f18a32a5', 'request_id': 'req-3f4e846d-a6d1-428e-8027-f3bfb2cde015'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.624424 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d46885820d4009aa6540a9f18a32a5', 'request_id': 'req-3f4e846d-a6d1-428e-8027-f3bfb2cde015'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.646148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] The token cache is being invalidate because project b5d46885820d4009aa6540a9f18a32a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:52.646630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.647226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f4e846d-a6d1-428e-8027-f3bfb2cde015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:52.649062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1133/2251] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:52 2022] DELETE /identity/v3/projects/b5d46885820d4009aa6540a9f18a32a5 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:52.656600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba5f3-e0b2-4bc7-9097-8e5e4e1e4756 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.657002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba5f3-e0b2-4bc7-9097-8e5e4e1e4756 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.657418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba5f3-e0b2-4bc7-9097-8e5e4e1e4756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.680703 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b85ba5f3-e0b2-4bc7-9097-8e5e4e1e4756 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:52.682015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b85ba5f3-e0b2-4bc7-9097-8e5e4e1e4756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:52.707653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1119/2252] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:52.722014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.722372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.722705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.753178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:52.754749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:52.776152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1120/2253] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:52.781053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:52.795668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53458be4-550d-4a8b-96e4-cb54c49f2d42 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:52.808705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1134/2254] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:52.812871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.813305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.813796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.818119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:52.820395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.820395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.820395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.821083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:52.821640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:52.824459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:52.851345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1121/2255] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:28:52.858330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:52.860472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:52.862863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:52.871883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1135/2256] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:52.877870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:52.885294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c145e-2201-4cfd-a3e1-1a6031e889d0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:52.888342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51532ea-ff01-4a5b-8030-85103866e724 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:52.893887 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:52.894078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:52.894903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.895019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:52.895183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.895460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:52.895573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.895951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:52.896234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:52.896545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:52.896810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:52.898217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:52.900565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:52.920530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1136/2257] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:28:52.926921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:52.928722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:52.945802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591780f-9581-430a-b34f-b65bd67dd715 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:52.956060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1122/2258] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:52.957515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.040564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.041256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.041537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.041831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.043012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.045063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.060569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbad87e-b42b-4a57-90a4-e6cf7561f356 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.066026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.066675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.066958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.067306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.068036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.070339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.085203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1137/2259] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:53.091901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.092616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1123/2260] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:52 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:53.098263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.113213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc0eb-f4b9-4409-bb71-5eec694c4677 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.166819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:53.190566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e8246-0771-4691-b4ac-8f84ac771180 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.196853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.197744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.198233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.198683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.199602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.202075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.215267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.215999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.216436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.216542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.217257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.219033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.231094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1124/2261] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:53.235534 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.243073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1138/2262] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:53 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:53.247587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.252150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b384284-7d05-4db2-924f-cdbc8a0685be admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.258512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.258813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb05a3-5cfc-4f83-a55b-63ff61f238ec admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.259530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.259999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.260401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.261545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.263736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.270748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.272128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.272615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.273125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.274496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.277374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.279989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1125/2263] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:53.285647 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.301502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1139/2264] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:53.302826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.305037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d514a0-9de5-47dc-a3a2-8d0aafcd06bc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.312493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5af475-bfd5-41e6-a9a5-6c5cfa84c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.313632 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5af475-bfd5-41e6-a9a5-6c5cfa84c948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.313917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5af475-bfd5-41e6-a9a5-6c5cfa84c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.314186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5af475-bfd5-41e6-a9a5-6c5cfa84c948 admin admin] PATH_INFO: `/v3/projects/9b439e97549748fa8336ab305a10eb70/users/913ee8d8e651410c85fa963d49f37d99/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.316921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21524587-6371-4924-a842-0a67604ed6d9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.324235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.324916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.325193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.325465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.326196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.328002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.335339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5af475-bfd5-41e6-a9a5-6c5cfa84c948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b439e97549748fa8336ab305a10eb70, user_id=913ee8d8e651410c85fa963d49f37d99, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.340553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1140/2265] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:53.346427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5af475-bfd5-41e6-a9a5-6c5cfa84c948 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.353304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.364587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1126/2266] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:53 2022] PUT /identity/v3/projects/9b439e97549748fa8336ab305a10eb70/users/913ee8d8e651410c85fa963d49f37d99/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:53.377180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.378341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.378341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.379708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a351145-0a59-413d-8e39-9e2b08c4ae0e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.416207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] MFA Rules not processed for user `913ee8d8e651410c85fa963d49f37d99`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:53.417464 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:53.457217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:53.481593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e41315-d1d7-4a80-8eb9-52f2ceff25be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.482500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41315-d1d7-4a80-8eb9-52f2ceff25be admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.482815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41315-d1d7-4a80-8eb9-52f2ceff25be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.483088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41315-d1d7-4a80-8eb9-52f2ceff25be admin admin] PATH_INFO: `/v3/projects/a48aa1226aa14887b99f6bdc88f91615/users/58b7a7f1bfd441d5a0771a959061d77f/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.507756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e41315-d1d7-4a80-8eb9-52f2ceff25be admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48aa1226aa14887b99f6bdc88f91615, user_id=58b7a7f1bfd441d5a0771a959061d77f, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.514849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1127/2267] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:53.518054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.518246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e41315-d1d7-4a80-8eb9-52f2ceff25be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.541446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1141/2268] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:53 2022] PUT /identity/v3/projects/a48aa1226aa14887b99f6bdc88f91615/users/58b7a7f1bfd441d5a0771a959061d77f/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:53.554700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190b5ad-2102-44d3-ae3a-d0838f3938d6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.557728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970995-1869-4043-852f-66fde8528aa4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.558093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970995-1869-4043-852f-66fde8528aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.558447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970995-1869-4043-852f-66fde8528aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.613049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-59970995-1869-4043-852f-66fde8528aa4 None None] MFA Rules not processed for user `58b7a7f1bfd441d5a0771a959061d77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:53.616029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-59970995-1869-4043-852f-66fde8528aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:53.643290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.644250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.644814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.645191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.646709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.649912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.682156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1128/2269] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:53 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:53.688032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.720319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1142/2270] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 15 15:28:53.736576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59970995-1869-4043-852f-66fde8528aa4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.738808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-59970995-1869-4043-852f-66fde8528aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:53.770223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:53.787105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4daa27-8dc3-4d59-97ac-5ad3e4ec1ff2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.792316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.792996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.793272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.793543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.794246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.796216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.811755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59970995-1869-4043-852f-66fde8528aa4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.820868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1129/2271] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:53 2022] POST /identity/v3/users => generated 301 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:28:53.825813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.835998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8df16-4106-484f-90d1-132193514ee1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.841428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.842884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.843176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.843456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.844191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.845805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.860939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1130/2272] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:53.866812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.879260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.879687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f615d-3eee-4acc-a42b-8c0aa6905c3d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:53.880023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.880302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.880571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.885346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:53.886382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:53.886878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:53.887333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] PATH_INFO: `/v3/projects/ce96039e68d24cf5a7839804d561608e/users/c40e586c93594750ac780e993901bf8f/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:53.911382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce96039e68d24cf5a7839804d561608e, user_id=c40e586c93594750ac780e993901bf8f, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.916784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.935317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1131/2273] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:53 2022] PUT /identity/v3/projects/ce96039e68d24cf5a7839804d561608e/users/c40e586c93594750ac780e993901bf8f/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:53.940380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:53.968484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:53.981119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:53.984060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:53.998617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1143/2274] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:28:54.039082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbbd33-73ab-415e-bb02-cd1bac91b21f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:54.044404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71529376-91ac-43f0-97e1-aa10dfda1503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:54.045105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71529376-91ac-43f0-97e1-aa10dfda1503 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:54.045390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71529376-91ac-43f0-97e1-aa10dfda1503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:54.045672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71529376-91ac-43f0-97e1-aa10dfda1503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:54.046413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71529376-91ac-43f0-97e1-aa10dfda1503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:54.048136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71529376-91ac-43f0-97e1-aa10dfda1503 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:54.067509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1132/2275] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:54.069046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:54.084749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962de0cb-fb6d-4a68-9ac3-120cd6cd789d service glance] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:54.092953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:54.096989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:54.096989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:54.096989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] PATH_INFO: `/v3/projects/ce96039e68d24cf5a7839804d561608e/users/c40e586c93594750ac780e993901bf8f/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:54.100174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce96039e68d24cf5a7839804d561608e, user_id=c40e586c93594750ac780e993901bf8f, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:54.108908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:54.130432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1144/2276] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:54 2022] PUT /identity/v3/projects/ce96039e68d24cf5a7839804d561608e/users/c40e586c93594750ac780e993901bf8f/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:54.144699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:54.145267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:54.145981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:54.202825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] MFA Rules not processed for user `c40e586c93594750ac780e993901bf8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:54.208617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:54.315312 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1133/2277] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:54.327071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:54.329529 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:54.409960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3437a0-6dcc-4f75-b735-70b9388a5648 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:54.414426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632fc40-b281-4f86-a81a-439d74b53a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:54.415133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632fc40-b281-4f86-a81a-439d74b53a49 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:54.415406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632fc40-b281-4f86-a81a-439d74b53a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:54.415673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632fc40-b281-4f86-a81a-439d74b53a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:54.432011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632fc40-b281-4f86-a81a-439d74b53a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:54.434815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632fc40-b281-4f86-a81a-439d74b53a49 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:54.452939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1145/2278] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:54.741358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:54.842595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452e827-9eb0-4f37-91fa-cf9ef93d267c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:54.848790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:54.849710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:54.849991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:54.850261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] PATH_INFO: `/v3/users/913ee8d8e651410c85fa963d49f37d99` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:54.851555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=913ee8d8e651410c85fa963d49f37d99)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:54.853882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:54.920998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:54.936886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913ee8d8e651410c85fa963d49f37d99'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:54.951446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913ee8d8e651410c85fa963d49f37d99'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:54.953285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913ee8d8e651410c85fa963d49f37d99'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:54.954337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913ee8d8e651410c85fa963d49f37d99'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:54.971341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913ee8d8e651410c85fa963d49f37d99'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:55.006807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1134/2279] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:54 2022] DELETE /identity/v3/users/913ee8d8e651410c85fa963d49f37d99 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:55.013783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:55.015433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:55.015433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:55.046614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:55.048604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:55.087588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:55.155491 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1146/2280] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:55.165607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:55.167129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:55.277919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9d4d1-e298-45fc-bee4-7bf4cbd9df7d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:55.283671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:55.284347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:55.284636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:55.285050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:55.300466 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:55.301920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:55.331759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1135/2281] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:55.779224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:55.784988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:55.819061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf30c91-ab2e-4c3c-a63e-0a328cf9a0e0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:55.837155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:55.839662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:55.840403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:55.841117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] PATH_INFO: `/v3/projects/9b439e97549748fa8336ab305a10eb70` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:55.843634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b439e97549748fa8336ab305a10eb70)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:55.848372 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:55.914234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b439e97549748fa8336ab305a10eb70', 'request_id': 'req-2f0beab2-2b4b-493a-a087-6f82d6014605'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:55.930241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b439e97549748fa8336ab305a10eb70', 'request_id': 'req-2f0beab2-2b4b-493a-a087-6f82d6014605'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:55.944230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b439e97549748fa8336ab305a10eb70', 'request_id': 'req-2f0beab2-2b4b-493a-a087-6f82d6014605'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:55.967685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] The token cache is being invalidate because project 9b439e97549748fa8336ab305a10eb70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:55.968229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:55.969302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:55.970987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1147/2282] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:55 2022] DELETE /identity/v3/projects/9b439e97549748fa8336ab305a10eb70 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:55.975431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:55.986884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711551f7-d29e-4736-949c-a93fce6392a5 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:56.062218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:56.063281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:56.063655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:56.064041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] PATH_INFO: `/v3/users/c40e586c93594750ac780e993901bf8f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:56.065529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c40e586c93594750ac780e993901bf8f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:56.068206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:56.152932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c40e586c93594750ac780e993901bf8f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:56.166955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c40e586c93594750ac780e993901bf8f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:56.167863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c40e586c93594750ac780e993901bf8f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:56.168696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c40e586c93594750ac780e993901bf8f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:56.178148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:56.182093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6e5feea3-feb4-4e94-b68a-41e156402327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c40e586c93594750ac780e993901bf8f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:56.218772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1136/2283] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:55 2022] DELETE /identity/v3/users/c40e586c93594750ac780e993901bf8f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:56.758708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:56.763297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:56.788603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0beab2-2b4b-493a-a087-6f82d6014605 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:56.896488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:56.898131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:56.898756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:56.899251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] PATH_INFO: `/v3/projects/ce96039e68d24cf5a7839804d561608e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:56.901312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce96039e68d24cf5a7839804d561608e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:56.905085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.001343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce96039e68d24cf5a7839804d561608e', 'request_id': 'req-60632408-7af1-4d23-99b4-07d15fba44e1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:57.003632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:57.011968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce96039e68d24cf5a7839804d561608e', 'request_id': 'req-60632408-7af1-4d23-99b4-07d15fba44e1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:57.026224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce96039e68d24cf5a7839804d561608e', 'request_id': 'req-60632408-7af1-4d23-99b4-07d15fba44e1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:57.045243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] The token cache is being invalidate because project ce96039e68d24cf5a7839804d561608e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:57.045707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:57.047064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-60632408-7af1-4d23-99b4-07d15fba44e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:57.049239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1148/2284] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:56 2022] DELETE /identity/v3/projects/ce96039e68d24cf5a7839804d561608e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:28:57.057063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.057864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.058003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.097625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:57.099556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:57.183317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1137/2285] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:57.192883 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.193178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.193457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.221035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:57.222439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:57.233123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1149/2286] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:57.237718 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.250928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a00c44-a4eb-45f8-aef8-1a7c9d77ba90 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:57.256331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:57.257226 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.258233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.258863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.259912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:57.260388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:57.262586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.283413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1138/2287] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:28:57.289241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.302743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dc624-6ff5-4871-bbc0-5e0a2606751c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:57.307975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:57.308713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.308994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.309278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.309982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:57.312002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.360558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1150/2288] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:57 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:57.365693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.383654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553b300-d1a5-4d29-a8f5-0e62b7300283 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:57.468084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:57.468789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.469110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.469442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.470155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:57.472340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.495559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1139/2289] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:57 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:57.500550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.514204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fc47c-d93f-41b6-a683-da25b17c9629 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:57.519515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:57.520216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.520500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.520775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.521641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:57.523422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.535227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1151/2290] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:28:57.541640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.558104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba64e2-b856-44df-bc7c-bfbdff9d9fea admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:57.565775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929e49b-4f2f-4b97-bc1f-82c690991cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:57.566654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929e49b-4f2f-4b97-bc1f-82c690991cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.566982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929e49b-4f2f-4b97-bc1f-82c690991cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.567382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929e49b-4f2f-4b97-bc1f-82c690991cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.568501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929e49b-4f2f-4b97-bc1f-82c690991cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:57.571417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929e49b-4f2f-4b97-bc1f-82c690991cbf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.583994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1140/2291] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:57.592639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.607865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5e472-b5b7-45b0-a277-4dc716648e03 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:57.615283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:57.616407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.616832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.617724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] PATH_INFO: `/v3/projects/dc5abed995224d27b6d3b3f0512e61cf/users/4ecc73a056d5407fbfc1ae63548d264f/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.641901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5abed995224d27b6d3b3f0512e61cf, user_id=4ecc73a056d5407fbfc1ae63548d264f, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:57.644984 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:57.662352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1152/2292] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:57 2022] PUT /identity/v3/projects/dc5abed995224d27b6d3b3f0512e61cf/users/4ecc73a056d5407fbfc1ae63548d264f/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:57.680189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:57.680531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:57.681227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:57.727267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] MFA Rules not processed for user `4ecc73a056d5407fbfc1ae63548d264f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:57.728665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:57.780699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:57.833590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1141/2293] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:57.848172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:57.850754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:58.017492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f50c7c-4c4e-4ad8-802d-c66cd752e90d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:58.024907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b6de1-b99e-4378-b400-529773682300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:58.026216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6de1-b99e-4378-b400-529773682300 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:58.026894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6de1-b99e-4378-b400-529773682300 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:58.028046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b6de1-b99e-4378-b400-529773682300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:58.051564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b6de1-b99e-4378-b400-529773682300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:58.054022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b6de1-b99e-4378-b400-529773682300 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:58.071660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1153/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:58.102566 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:58.218850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678609-38fd-4f5e-8372-f4930961a1a2 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:58.227528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:58.229107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:58.229629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:58.230181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] PATH_INFO: `/v3/users/4ecc73a056d5407fbfc1ae63548d264f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:58.232384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ecc73a056d5407fbfc1ae63548d264f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:58.235824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:58.306462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ecc73a056d5407fbfc1ae63548d264f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:58.316172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ecc73a056d5407fbfc1ae63548d264f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:58.316978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ecc73a056d5407fbfc1ae63548d264f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:58.317895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ecc73a056d5407fbfc1ae63548d264f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:58.330374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ecc73a056d5407fbfc1ae63548d264f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:58.360901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1142/2295] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:28:58 2022] DELETE /identity/v3/users/4ecc73a056d5407fbfc1ae63548d264f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:58.368728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:58.369203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:58.369649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:58.395670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:58.397160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:58.478837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1154/2296] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:58.493832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:58.495664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:58.561662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45a37f-b8fa-4b08-aa75-4d651574cf25 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:58.565563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5586e-d8db-4e0f-839e-06a58716a450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:58.566235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5586e-d8db-4e0f-839e-06a58716a450 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:58.566561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5586e-d8db-4e0f-839e-06a58716a450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:58.566831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5586e-d8db-4e0f-839e-06a58716a450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:58.575865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:58.584527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5586e-d8db-4e0f-839e-06a58716a450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:58.587230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5586e-d8db-4e0f-839e-06a58716a450 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:58.603615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1143/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:28:58.963299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:58.964953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:58.978113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbf289-439b-4b27-a3cf-513fc26b95b7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:58.983314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:58.984221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:58.984504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:58.984830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] PATH_INFO: `/v3/projects/dc5abed995224d27b6d3b3f0512e61cf` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:58.986215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5abed995224d27b6d3b3f0512e61cf)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:58.988931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.011598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:28:59.060442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5abed995224d27b6d3b3f0512e61cf', 'request_id': 'req-729cac10-88aa-4299-9007-8fab8be50dff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:59.071936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5abed995224d27b6d3b3f0512e61cf', 'request_id': 'req-729cac10-88aa-4299-9007-8fab8be50dff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:59.089031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5abed995224d27b6d3b3f0512e61cf', 'request_id': 'req-729cac10-88aa-4299-9007-8fab8be50dff'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:59.102099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] The token cache is being invalidate because project dc5abed995224d27b6d3b3f0512e61cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:28:59.102408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:59.103094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-729cac10-88aa-4299-9007-8fab8be50dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:28:59.104672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1155/2298] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:28:58 2022] DELETE /identity/v3/projects/dc5abed995224d27b6d3b3f0512e61cf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:59.109805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.110350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.110350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.136786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:59.141268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:59.262989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1144/2299] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:28:59.272040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.272534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.273013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.308642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:59.315702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:59.328637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1156/2300] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:59.333631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.348683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd67d8-9d7e-4845-822a-195d603dd4eb None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:59.356625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:59.357835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.358348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.358755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.360019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:28:59.361321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:59.362322 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.430711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1145/2301] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:28:59.436016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.475260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe28005-e915-4871-82e0-311c66b37bf8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:59.485362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:59.486048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.486330 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.486831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.487651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:59.489726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.528333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1157/2302] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:28:59 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:59.533336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.549587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9c8a5-116a-4fa8-829b-aae3a5947b11 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:59.627161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:59.627844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.628128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.628404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.629132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:59.631112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.658615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1146/2303] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:28:59 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:28:59.661978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.673779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63fb74-3a99-477b-a079-e716d006c4a8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:59.678859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:59.679616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.679938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.680313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.681516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:59.684116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.698777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1158/2304] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:59.703543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.717294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf199d5-5ab5-4420-bde7-2df13283d9ea admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:59.724041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9bfa-4da7-4417-ace7-09851b97bbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:59.724995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9bfa-4da7-4417-ace7-09851b97bbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.724995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9bfa-4da7-4417-ace7-09851b97bbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.724995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9bfa-4da7-4417-ace7-09851b97bbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.726015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9bfa-4da7-4417-ace7-09851b97bbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:59.728049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9bfa-4da7-4417-ace7-09851b97bbc8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.741197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1147/2305] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:28:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:28:59.747582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.766053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5159da-3dc6-4d9c-91a3-eeec6d0ef64d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:28:59.772575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:28:59.773726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.774006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.774280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] PATH_INFO: `/v3/projects/7987f63707dc436c837b81c9c7e32888/users/8b113432fd9d4e4dafc74874a12364a6/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.794763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7987f63707dc436c837b81c9c7e32888, user_id=8b113432fd9d4e4dafc74874a12364a6, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:28:59.797594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:28:59.815484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1159/2306] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:28:59 2022] PUT /identity/v3/projects/7987f63707dc436c837b81c9c7e32888/users/8b113432fd9d4e4dafc74874a12364a6/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:28:59.822298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05f200-9f43-4470-be43-8ebe156a2f04 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:28:59.822800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05f200-9f43-4470-be43-8ebe156a2f04 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:28:59.823192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05f200-9f43-4470-be43-8ebe156a2f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:28:59.847927 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fd05f200-9f43-4470-be43-8ebe156a2f04 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:28:59.849426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fd05f200-9f43-4470-be43-8ebe156a2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:28:59.951845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1148/2307] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:28:59.967567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:28:59.971763 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:00.070119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfc1aa-595f-4128-93a6-b6dc83fd0957 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:00.077045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:00.077740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:00.078032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:00.078312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:00.094560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:00.096780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:00.109152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1160/2308] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:29:01.339725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137468a-7882-4941-97c7-8e86f654507c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:01.339725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137468a-7882-4941-97c7-8e86f654507c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:01.340951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137468a-7882-4941-97c7-8e86f654507c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:01.350040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:01.395378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c137468a-7882-4941-97c7-8e86f654507c None None] MFA Rules not processed for user `8b113432fd9d4e4dafc74874a12364a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:01.397382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c137468a-7882-4941-97c7-8e86f654507c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:01.512004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1149/2309] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:29:01.525189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:01.526909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:01.599760 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:01.628207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dabc2-6c71-41a6-8635-731b4f88e876 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:01.633446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7f791-0f80-437f-b046-4b2f5116b433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:01.634343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f791-0f80-437f-b046-4b2f5116b433 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:01.634817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f791-0f80-437f-b046-4b2f5116b433 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:01.635104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7f791-0f80-437f-b046-4b2f5116b433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:01.647346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f791-0f80-437f-b046-4b2f5116b433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:01.650132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7f791-0f80-437f-b046-4b2f5116b433 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:01.663787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1161/2310] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:29:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:29:01.790101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137468a-7882-4941-97c7-8e86f654507c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:01.791960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c137468a-7882-4941-97c7-8e86f654507c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:01.811206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137468a-7882-4941-97c7-8e86f654507c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:01.815937 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:01.816867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:01.817147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:01.817425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] PATH_INFO: `/v3/users/c2101f9a4f394bd79239cad2f0b83ea6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:01.818705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2101f9a4f394bd79239cad2f0b83ea6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:01.821601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:01.904895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2101f9a4f394bd79239cad2f0b83ea6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:01.917360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2101f9a4f394bd79239cad2f0b83ea6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:01.918436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2101f9a4f394bd79239cad2f0b83ea6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:01.919468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2101f9a4f394bd79239cad2f0b83ea6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:01.935031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2101f9a4f394bd79239cad2f0b83ea6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:01.977516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1150/2311] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:29:01 2022] DELETE /identity/v3/users/c2101f9a4f394bd79239cad2f0b83ea6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:29:01.984053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:01.984641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:01.985013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:02.022822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:02.026209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:02.139742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1162/2312] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:29:02.158165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:02.160923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:02.240051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab797f8-b0dd-4de5-9a56-c92839a61855 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:02.244137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff02f0-f4ad-4812-ace6-20dda6de0e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:02.244821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff02f0-f4ad-4812-ace6-20dda6de0e89 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:02.245108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff02f0-f4ad-4812-ace6-20dda6de0e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:02.245389 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff02f0-f4ad-4812-ace6-20dda6de0e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:02.257252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff02f0-f4ad-4812-ace6-20dda6de0e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:02.260159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff02f0-f4ad-4812-ace6-20dda6de0e89 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:02.272761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1151/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:29:02.742475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:02.744924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:02.762566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cf335-c995-456b-a645-1974af35d7fe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:02.784634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:02.785486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:02.785776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:02.786058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] PATH_INFO: `/v3/projects/1c2735bba83f4ea19182c12c13b65601` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:02.787373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2735bba83f4ea19182c12c13b65601)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:02.789588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:02.859776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2735bba83f4ea19182c12c13b65601', 'request_id': 'req-ffbf05de-9910-42cf-94f9-a1032001c43a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:02.869664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2735bba83f4ea19182c12c13b65601', 'request_id': 'req-ffbf05de-9910-42cf-94f9-a1032001c43a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:02.878057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2735bba83f4ea19182c12c13b65601', 'request_id': 'req-ffbf05de-9910-42cf-94f9-a1032001c43a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:02.891545 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] The token cache is being invalidate because project 1c2735bba83f4ea19182c12c13b65601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:29:02.891829 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:02.892503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:02.894085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1163/2314] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:29:02 2022] DELETE /identity/v3/projects/1c2735bba83f4ea19182c12c13b65601 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:29:04.277286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:04.277867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:04.277867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:04.292837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:04.300529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] MFA Rules not processed for user `8b113432fd9d4e4dafc74874a12364a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:04.302705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:04.380491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1152/2315] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:29:12.888192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:14.373002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=22 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:15.454702 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:15.457041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:15.548173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf05de-9910-42cf-94f9-a1032001c43a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:15.556209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:15.557338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:15.557845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:15.558369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:15.561568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:15.578582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:15.580671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a400fe3-091d-410f-b68b-24ea98388d61 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:15.594866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1164/2316] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:29:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:29:22.042705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:22.044288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:22.151485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedaf18-27e8-4c8d-906c-1b02f5124f63 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:22.159478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:22.161037 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:22.161513 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:22.162015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] PATH_INFO: `/v3/users/260ee759924245a1bbc198f24aca76fc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:22.164430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=260ee759924245a1bbc198f24aca76fc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:22.168289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:22.238073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '260ee759924245a1bbc198f24aca76fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:22.247506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '260ee759924245a1bbc198f24aca76fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:22.248288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '260ee759924245a1bbc198f24aca76fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:22.249291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '260ee759924245a1bbc198f24aca76fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:22.257842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '260ee759924245a1bbc198f24aca76fc'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:22.277898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1153/2317] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:29:22 2022] DELETE /identity/v3/users/260ee759924245a1bbc198f24aca76fc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:29:22.282871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:22.283526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:22.283526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:22.304550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:22.305974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:22.384974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1165/2318] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:29:22.397225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:22.399422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:22.480621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195ec59-3cf2-47f0-9841-dae3b0155558 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:22.485450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef35398-082c-4f8c-ad73-095b4bd64833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:22.486628 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef35398-082c-4f8c-ad73-095b4bd64833 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:22.486944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef35398-082c-4f8c-ad73-095b4bd64833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:22.487268 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef35398-082c-4f8c-ad73-095b4bd64833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:22.504474 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef35398-082c-4f8c-ad73-095b4bd64833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:22.507742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef35398-082c-4f8c-ad73-095b4bd64833 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:22.546258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1154/2319] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:29:23.218144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:23.220125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:23.233931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96a6c0-eab7-4494-80ac-01b2b6c41211 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:23.240046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:23.241266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.241606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.241948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] PATH_INFO: `/v3/projects/d9d3a797588e467aaca0a843a39d7ae1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.243464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d3a797588e467aaca0a843a39d7ae1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:23.245673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:23.351270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d3a797588e467aaca0a843a39d7ae1', 'request_id': 'req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:23.363809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d3a797588e467aaca0a843a39d7ae1', 'request_id': 'req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:23.377615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d3a797588e467aaca0a843a39d7ae1', 'request_id': 'req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:23.402799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] The token cache is being invalidate because project d9d3a797588e467aaca0a843a39d7ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:29:23.403474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:23.404340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-7ffed315-9d76-4d65-9d9d-11c4e3ac6e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:29:23.406660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1166/2320] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:29:23 2022] DELETE /identity/v3/projects/d9d3a797588e467aaca0a843a39d7ae1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:29:23.414369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.415182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.416448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.449990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:23.451874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:23.483502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:23.575777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1155/2321] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:29:23.585940 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.586390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.586923 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.617805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:23.619127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:23.629108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1167/2322] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:29:23.633995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:23.654736 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f41b32-b057-4e22-ae6c-9bab99afc5c1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:23.661956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:23.663644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.664197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.664571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.665453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:29:23.665855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:23.667868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:23.692740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1156/2323] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:29:23.697413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:23.716482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f23b1-0bcb-4582-977f-f5fcb427d329 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:23.725816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:23.726991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.727561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.727901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.728969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:23.732568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:23.738487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:23.779897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1168/2324] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:29:23 2022] POST /identity/v3/projects => generated 391 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:29:23.790858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:23.803951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc219b87-4a14-4337-9f65-207db41b234e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:23.883301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:23.884209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.884575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.884988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.885783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:23.888227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:23.911326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1157/2325] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:29:23 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:29:23.917096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:23.935015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a53aa5-d3ac-4f53-a693-f4fd7fa49859 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:23.940022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:23.940704 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.940993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.941267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.942009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:23.944683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:23.958622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1169/2326] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:29:23.963285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:23.976401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecb57c-532c-41e0-b369-60bee2ea73a3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:23.982150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aedd1b2-c070-4a3f-b410-7e468bf682c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:23.983375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd1b2-c070-4a3f-b410-7e468bf682c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:23.983991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd1b2-c070-4a3f-b410-7e468bf682c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:23.984390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aedd1b2-c070-4a3f-b410-7e468bf682c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:23.985703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedd1b2-c070-4a3f-b410-7e468bf682c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:23.988121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aedd1b2-c070-4a3f-b410-7e468bf682c2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:24.004157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1158/2327] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:29:24.010556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:24.027239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6dcc4-20c7-4eaf-a133-158e0776a151 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:24.032658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:24.033488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:24.033774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:24.034049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] PATH_INFO: `/v3/projects/3d100774fa0b468d86ecfd9dfa024ca0/users/c5bd28e89677438a82543821453317a1/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:24.052161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d100774fa0b468d86ecfd9dfa024ca0, user_id=c5bd28e89677438a82543821453317a1, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:24.055525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:24.076240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1170/2328] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:29:24 2022] PUT /identity/v3/projects/3d100774fa0b468d86ecfd9dfa024ca0/users/c5bd28e89677438a82543821453317a1/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:29:24.089223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca0901-d644-43ff-ba8a-106f1b81d906 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:24.089562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca0901-d644-43ff-ba8a-106f1b81d906 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:24.089906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca0901-d644-43ff-ba8a-106f1b81d906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:24.128800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-70ca0901-d644-43ff-ba8a-106f1b81d906 None None] MFA Rules not processed for user `c5bd28e89677438a82543821453317a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:24.130918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-70ca0901-d644-43ff-ba8a-106f1b81d906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:24.245828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1159/2329] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:29:24.259544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:29:24.262761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:24.352634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76922603-2d54-4343-9fd8-41be4002fb30 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:29:24.356561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:29:24.357271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:24.357552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:24.357827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:24.369246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:29:24.371032 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:29:24.381730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1171/2330] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:29:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:29:28.400003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3cc1d-f6e4-495a-a67c-7d6d95f64f17 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:29:28.400609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3cc1d-f6e4-495a-a67c-7d6d95f64f17 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:29:28.400609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3cc1d-f6e4-495a-a67c-7d6d95f64f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:29:28.419094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a7d3cc1d-f6e4-495a-a67c-7d6d95f64f17 None None] MFA Rules not processed for user `c1a1af21d3f94b70a830b14200619f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:29:28.420311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a7d3cc1d-f6e4-495a-a67c-7d6d95f64f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:29:28.433590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:28.507053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1160/2331] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:29:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:29:34.376448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:29:38.495637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=17 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:01.519198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:01.666841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686510-bb92-4ebd-80de-3f995b6046f1 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:01.680202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:01.681293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:01.681670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:01.682095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] PATH_INFO: `/v3/users/c5bd28e89677438a82543821453317a1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:01.683843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5bd28e89677438a82543821453317a1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:01.686195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:01.767224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5bd28e89677438a82543821453317a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:01.777902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5bd28e89677438a82543821453317a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:01.791375 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5bd28e89677438a82543821453317a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:01.792206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5bd28e89677438a82543821453317a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:01.793191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5bd28e89677438a82543821453317a1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:01.824422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1172/2332] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:30:01 2022] DELETE /identity/v3/users/c5bd28e89677438a82543821453317a1 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:01.831877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:01.832705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:01.832848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:01.863275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:01.864689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:01.974198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1161/2333] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:01.986767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:01.989315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:02.081106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a9b3c-57cc-4a8d-a09d-04a1786291c5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:02.085186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0424-7a2e-47d6-a8d5-2b3153062034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:02.085907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0424-7a2e-47d6-a8d5-2b3153062034 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:02.086187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0424-7a2e-47d6-a8d5-2b3153062034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:02.086572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0424-7a2e-47d6-a8d5-2b3153062034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:02.100505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0424-7a2e-47d6-a8d5-2b3153062034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:02.102482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0424-7a2e-47d6-a8d5-2b3153062034 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:02.117973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1173/2334] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:30:02.480448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:02.482756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:02.502694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e456c-253c-4cda-8159-60dcad90f7d7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:02.507991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:02.509766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:02.510262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:02.510782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] PATH_INFO: `/v3/projects/3d100774fa0b468d86ecfd9dfa024ca0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:02.512643 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d100774fa0b468d86ecfd9dfa024ca0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:02.516561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:02.588297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d100774fa0b468d86ecfd9dfa024ca0', 'request_id': 'req-a440db8e-6711-4421-a1f9-3c967471737c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:02.595695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d100774fa0b468d86ecfd9dfa024ca0', 'request_id': 'req-a440db8e-6711-4421-a1f9-3c967471737c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:02.602922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d100774fa0b468d86ecfd9dfa024ca0', 'request_id': 'req-a440db8e-6711-4421-a1f9-3c967471737c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:02.624562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] The token cache is being invalidate because project 3d100774fa0b468d86ecfd9dfa024ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:30:02.625303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:02.626482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a440db8e-6711-4421-a1f9-3c967471737c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:02.629785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1162/2335] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:30:02 2022] DELETE /identity/v3/projects/3d100774fa0b468d86ecfd9dfa024ca0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:30:02.637289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:02.637830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:02.638218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:02.663443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:02.664701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:02.693007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:02.765448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1174/2336] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:30:02.775098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:02.775520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:02.775900 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:02.802651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:02.803991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:02.813591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1163/2337] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:02.818786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:02.835784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a4749-de1e-4cff-bb24-ca025ffaa712 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:02.843011 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:02.844074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:02.844526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:02.844983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:02.846344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:30:02.846827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:02.849635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:02.871688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1175/2338] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:30:02.876783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:02.888599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeebaf8-30c7-4e3f-ad63-af09a0c50db7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:02.894094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:02.894934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:02.895252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:02.895560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:02.896300 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:02.898390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:02.932676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1164/2339] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:30:02 2022] POST /identity/v3/projects => generated 397 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:30:02.938281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:02.957171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adace150-0d96-46a5-a22a-e67b4c03d1ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:03.050054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:03.050847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.051129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.051412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.052129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:03.054084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:03.091876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1176/2340] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:30:02 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:30:03.093762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:03.112753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f596822-adbb-434d-b13e-0cd4bd7232c9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:03.120434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:03.121602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.122085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.122601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.124009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:03.127285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:03.146139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1165/2341] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:30:03.150345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:03.169802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa070d-31ce-4cfb-aa8d-105d84bc0555 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:03.178241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191374f-edb2-4ad5-a3a5-8bfab529ac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:03.179508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191374f-edb2-4ad5-a3a5-8bfab529ac14 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.179980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191374f-edb2-4ad5-a3a5-8bfab529ac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.180446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191374f-edb2-4ad5-a3a5-8bfab529ac14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.181697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191374f-edb2-4ad5-a3a5-8bfab529ac14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:03.183398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b191374f-edb2-4ad5-a3a5-8bfab529ac14 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:03.195486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1177/2342] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:30:03.200240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:03.217795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24194edc-1c60-4c51-b574-b4d92326aec5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:03.225725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:03.227324 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.227785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.228260 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] PATH_INFO: `/v3/projects/96aaa7f7f881403daee6528320fcae40/users/b6317b1a931d461fbc6a3aa0a3d47caf/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.254412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96aaa7f7f881403daee6528320fcae40, user_id=b6317b1a931d461fbc6a3aa0a3d47caf, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:03.257167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:03.284106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1166/2343] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:30:03 2022] PUT /identity/v3/projects/96aaa7f7f881403daee6528320fcae40/users/b6317b1a931d461fbc6a3aa0a3d47caf/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:03.298809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd5c5-4227-4609-be09-0f6d4e661c5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.299156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd5c5-4227-4609-be09-0f6d4e661c5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.299900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341fd5c5-4227-4609-be09-0f6d4e661c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.346744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-341fd5c5-4227-4609-be09-0f6d4e661c5a None None] MFA Rules not processed for user `b6317b1a931d461fbc6a3aa0a3d47caf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:03.349632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-341fd5c5-4227-4609-be09-0f6d4e661c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:03.487463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1178/2344] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:03.503567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:03.506957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:03.549558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:03.609754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71400d-3bd3-460f-9834-a67b61046b1b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:03.616340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:03.617583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.618164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.618797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.636053 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:03.638091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:03.652847 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1167/2345] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:30:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:30:03.873430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5737-279b-42cc-9c46-bbf3cc388619 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:03.874070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5737-279b-42cc-9c46-bbf3cc388619 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:03.874432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf5737-279b-42cc-9c46-bbf3cc388619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:03.914718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e6cf5737-279b-42cc-9c46-bbf3cc388619 None None] MFA Rules not processed for user `8b113432fd9d4e4dafc74874a12364a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:03.916404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e6cf5737-279b-42cc-9c46-bbf3cc388619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:03.951425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:04.023830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1179/2346] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:13.646272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=17 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:14.009634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=17 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:17.561215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:17.670449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331178a8-867f-4a51-97b0-ed49706e8aa8 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:17.678728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:17.680318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:17.680689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:17.681302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] PATH_INFO: `/v3/users/b6317b1a931d461fbc6a3aa0a3d47caf` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:17.683568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6317b1a931d461fbc6a3aa0a3d47caf)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:17.687512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:17.750229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6317b1a931d461fbc6a3aa0a3d47caf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:17.763227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6317b1a931d461fbc6a3aa0a3d47caf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:17.763909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6317b1a931d461fbc6a3aa0a3d47caf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:17.764650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6317b1a931d461fbc6a3aa0a3d47caf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:17.774460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6317b1a931d461fbc6a3aa0a3d47caf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:17.794333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1168/2347] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:30:17 2022] DELETE /identity/v3/users/b6317b1a931d461fbc6a3aa0a3d47caf => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:17.800215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:17.801001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:17.801001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:17.828616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:17.830751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:17.925486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1180/2348] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:17.944560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:17.947029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:18.046822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7bea93-391c-4ebb-9999-8b7281592724 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:18.053662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:18.054836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:18.055313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:18.055795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:18.071618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:18.074232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:18.090893 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1169/2349] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:30:18.453492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:18.456953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:18.480665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:18.482741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:18.504797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311ccd-2b40-4d88-b978-e1e9bec2e314 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:18.506880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0b1cb-1dba-4790-8418-57f5c786ccb2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:18.520275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:18.524485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:18.524485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:18.524485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] PATH_INFO: `/v3/users/58b7a7f1bfd441d5a0771a959061d77f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:18.527605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b7a7f1bfd441d5a0771a959061d77f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:18.529749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:18.529916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:18.531299 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:18.532165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:18.532165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] PATH_INFO: `/v3/projects/96aaa7f7f881403daee6528320fcae40` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:18.536461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96aaa7f7f881403daee6528320fcae40)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:18.540270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:18.563271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:18.643468 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b7a7f1bfd441d5a0771a959061d77f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.649521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96aaa7f7f881403daee6528320fcae40', 'request_id': 'req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.666164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b7a7f1bfd441d5a0771a959061d77f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.668923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b7a7f1bfd441d5a0771a959061d77f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.671485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b7a7f1bfd441d5a0771a959061d77f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.676115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96aaa7f7f881403daee6528320fcae40', 'request_id': 'req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.684884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96aaa7f7f881403daee6528320fcae40', 'request_id': 'req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.687087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b7a7f1bfd441d5a0771a959061d77f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.715127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] The token cache is being invalidate because project 96aaa7f7f881403daee6528320fcae40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:30:18.715789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.716781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6cd75ea6-05c5-4fe0-b156-9fee5c07eb80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:18.719846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1181/2350] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:30:18 2022] DELETE /identity/v3/projects/96aaa7f7f881403daee6528320fcae40 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:30:18.742401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1170/2351] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:30:18 2022] DELETE /identity/v3/users/58b7a7f1bfd441d5a0771a959061d77f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:18.744742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:18.745389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:18.746618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:18.785257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:18.786742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:18.919070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1182/2352] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:30:18.934635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:18.936917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:19.056727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076205e-5185-47d4-8596-8921326049e4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:19.063577 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd7ea1-c2b1-450f-808f-82c2c7ace6dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:19.064563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd7ea1-c2b1-450f-808f-82c2c7ace6dd service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:19.064846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd7ea1-c2b1-450f-808f-82c2c7ace6dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:19.065216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd7ea1-c2b1-450f-808f-82c2c7ace6dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:19.083748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd7ea1-c2b1-450f-808f-82c2c7ace6dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:19.086232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd7ea1-c2b1-450f-808f-82c2c7ace6dd service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:19.102270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1171/2353] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:30:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:30:19.591503 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:19.596206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:19.619740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166d00-3a93-43fd-9ba2-7496297f7ea6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:19.625121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:19.626284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:19.626719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:19.627312 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] PATH_INFO: `/v3/projects/a48aa1226aa14887b99f6bdc88f91615` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:19.628822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48aa1226aa14887b99f6bdc88f91615)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:19.631012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:19.675348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:19.748446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48aa1226aa14887b99f6bdc88f91615', 'request_id': 'req-146382a1-55ad-4855-9e0b-77b2bb84c85f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:19.760699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48aa1226aa14887b99f6bdc88f91615', 'request_id': 'req-146382a1-55ad-4855-9e0b-77b2bb84c85f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:19.776256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48aa1226aa14887b99f6bdc88f91615', 'request_id': 'req-146382a1-55ad-4855-9e0b-77b2bb84c85f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:19.794233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] The token cache is being invalidate because project a48aa1226aa14887b99f6bdc88f91615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:30:19.794725 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:19.800749 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:19.803715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1183/2354] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:30:19 2022] DELETE /identity/v3/projects/a48aa1226aa14887b99f6bdc88f91615 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:29.097511 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:29.792033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:36.177355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f375-413e-48c8-86b4-15782649ad3f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:36.178101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f375-413e-48c8-86b4-15782649ad3f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:36.179031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea2f375-413e-48c8-86b4-15782649ad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:36.204313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-6ea2f375-413e-48c8-86b4-15782649ad3f None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:36.205686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6ea2f375-413e-48c8-86b4-15782649ad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:36.277009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1172/2355] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:30:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:36.320245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:36.321740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:36.398487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146382a1-55ad-4855-9e0b-77b2bb84c85f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:36.404094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:36.404747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:36.405025 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:36.405296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:36.417817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:36.419439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:36.428264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1184/2356] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:30:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:30:46.263422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=25 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:46.424740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=32 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:53.430132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:53.430990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:53.430990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:53.467916 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:53.469479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:53.476918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1173/2357] 10.222.0.40 () {58 vars in 967 bytes} [Fri Jul 15 15:30:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:53.489607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:53.501383 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787bed6-fe4f-44bc-822d-cf2b4aef526e service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:53.507461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:53.508232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:53.508631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:53.509005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:53.521185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:53.523890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:53.535687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1185/2358] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:30:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:30:55.611487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:55.613458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:55.699127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788fd53-52e1-46df-8b44-40bf99b8ba8e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:55.706035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:55.707337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:55.707772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:55.708208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] PATH_INFO: `/v3/users/8b113432fd9d4e4dafc74874a12364a6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:55.710405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b113432fd9d4e4dafc74874a12364a6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:55.713336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:55.768556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b113432fd9d4e4dafc74874a12364a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:55.777526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b113432fd9d4e4dafc74874a12364a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:55.778313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b113432fd9d4e4dafc74874a12364a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:55.778906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b113432fd9d4e4dafc74874a12364a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:55.787266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-a7bfdada-eedd-4a26-bebe-6132534eaa9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b113432fd9d4e4dafc74874a12364a6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:55.807657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1174/2359] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:30:55 2022] DELETE /identity/v3/users/8b113432fd9d4e4dafc74874a12364a6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:30:55.999179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.014833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223140a-3111-4ae2-baa3-4f8027219ae6 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.089039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.089913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.090196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.090467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] PATH_INFO: `/v3/projects/7987f63707dc436c837b81c9c7e32888` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.092033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7987f63707dc436c837b81c9c7e32888)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.094198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.145448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7987f63707dc436c837b81c9c7e32888', 'request_id': 'req-e4adfa78-c501-4449-8914-45d7bc9fcb8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:56.154183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7987f63707dc436c837b81c9c7e32888', 'request_id': 'req-e4adfa78-c501-4449-8914-45d7bc9fcb8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:56.160487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7987f63707dc436c837b81c9c7e32888', 'request_id': 'req-e4adfa78-c501-4449-8914-45d7bc9fcb8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:56.172874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] The token cache is being invalidate because project 7987f63707dc436c837b81c9c7e32888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:30:56.173167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:56.173917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e4adfa78-c501-4449-8914-45d7bc9fcb8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:30:56.175662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1186/2360] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:30:55 2022] DELETE /identity/v3/projects/7987f63707dc436c837b81c9c7e32888 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:56.180791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.181079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.181355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.200233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:56.201815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:56.286011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1175/2361] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:56.294914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.295206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.295486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.319160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:56.320403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:56.331117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1187/2362] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:56.336232 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.346568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110733d0-12b3-4c03-8d15-65d74b4afeab None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.351495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.352131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.352347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.352666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.353407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:30:56.353791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.355332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.366913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:56.370797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1176/2363] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:30:56.375333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.386701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971ece1-30ba-4138-a664-276c22b2fb70 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.393112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.393810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.394091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.394365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.395112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.397054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.426531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1188/2364] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:30:56 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:30:56.433316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.455500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711faa13-d90d-4a96-8178-7bc77420cfb9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.515923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.516720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.517003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.517276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.517968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.519918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.549752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1177/2365] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:30:56 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:30:56.551267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.563664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9e842-b903-4f88-8acd-89633293d930 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.568467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.569128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.569439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.569716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.570433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.572060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.596559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1189/2366] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:30:56.601629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.613318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41148a7-4ef0-4ba6-9e09-5b2ba2f2f7fa admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.618719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de10a8-67cb-4552-88fc-f05ce82ea2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.619402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de10a8-67cb-4552-88fc-f05ce82ea2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.619689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de10a8-67cb-4552-88fc-f05ce82ea2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.619969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de10a8-67cb-4552-88fc-f05ce82ea2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.620711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de10a8-67cb-4552-88fc-f05ce82ea2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.622332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de10a8-67cb-4552-88fc-f05ce82ea2f8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.633283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1178/2367] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:30:56.637931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.653277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107a16-cad1-4944-94ff-8fccacc21109 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.658930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.659943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.660298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.660603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] PATH_INFO: `/v3/projects/6fd8c6fef93d4313bb7894175242fd8c/users/28669cdf427a4012a766df06d598e54d/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.691910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd8c6fef93d4313bb7894175242fd8c, user_id=28669cdf427a4012a766df06d598e54d, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.694651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.708455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1190/2368] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:30:56 2022] PUT /identity/v3/projects/6fd8c6fef93d4313bb7894175242fd8c/users/28669cdf427a4012a766df06d598e54d/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:30:56.715299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fad74-4f84-4976-9321-a40e87b00472 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.715712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fad74-4f84-4976-9321-a40e87b00472 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.716066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fad74-4f84-4976-9321-a40e87b00472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.739724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b76fad74-4f84-4976-9321-a40e87b00472 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:56.740773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b76fad74-4f84-4976-9321-a40e87b00472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:56.814261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1179/2369] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:56.824942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:56.827045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:56.842082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:30:56.889963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c31cb-3f02-4e25-bf92-fd95cc7eb81d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:56.894044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:56.894731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:56.895034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:56.895320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:56.905885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:56.907444 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:56.917715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1191/2370] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:30:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:30:57.638007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379378a3-2d87-4809-9570-36ca294cedcb None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:57.639047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379378a3-2d87-4809-9570-36ca294cedcb None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:57.639666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379378a3-2d87-4809-9570-36ca294cedcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:57.671971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-379378a3-2d87-4809-9570-36ca294cedcb None None] MFA Rules not processed for user `28669cdf427a4012a766df06d598e54d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:57.673256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-379378a3-2d87-4809-9570-36ca294cedcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:57.779943 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1180/2371] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:30:57.797364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:30:57.799276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:57.865389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c789bb2-36f9-47f6-9f6a-7e5435d20327 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:30:57.869126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:30:57.869788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:57.870071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:57.870347 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:57.880991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:30:57.883321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:30:57.894655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1192/2372] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:30:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:30:59.310675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:30:59.311270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:30:59.311270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:30:59.330253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] MFA Rules not processed for user `28669cdf427a4012a766df06d598e54d`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:30:59.332235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:30:59.340063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1181/2373] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:30:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:31:07.888229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=46 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:09.324278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:12.987166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:12.997743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea2075-d1c7-4b88-9415-086b9974478b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:13.001477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:13.002136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:13.002415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:13.003070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:13.004932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:13.097827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:13.099371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:13.112719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1193/2374] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:31:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:31:16.555949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:16.567321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61709e6-95dc-4408-ba89-cf8a9c7b0e4b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:16.571604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:16.572270 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:16.572557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:16.572834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:16.575051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:16.591144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:16.592815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:16.609483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1182/2375] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:31:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:31:22.033492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:22.035145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:22.051258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1d154-1803-4761-a295-b7b36b63b8fb service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:22.060036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:22.060859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:22.061038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:22.061423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:22.063567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:22.150351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:22.152024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd45a4-c596-4416-b60a-c153cddae665 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:22.161583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1194/2376] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:31:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:31:26.602429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:30.110614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:30.112225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:30.131018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2086e-008d-473c-93b0-c8ab83ae5112 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:30.137969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:30.139026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:30.139523 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:30.139981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:30.142589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:30.229869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:30.232009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:30.243795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1183/2377] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:31:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:31:32.157166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=62 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:40.237800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=28 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:31:51.290615 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.291224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.291756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.314220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:31:51.315571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:51.397509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1195/2378] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:31:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:31:51.432623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:31:51.436212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:31:51.540235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f1e8b-b95d-4c7e-b06b-8b39a3a9aeb1 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:31:51.544322 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:31:51.545677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:31:51.546107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:31:51.546544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:31:51.567882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:31:51.569598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:31:51.581822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1184/2379] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:32:01.387185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:01.575919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=32 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:03.434791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:03.446307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90704c-ba40-4868-8c7d-353cdde58d2d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:03.451802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:03.452473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:03.452755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:03.453031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:03.455427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:03.468780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:03.470352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99678bcf-846a-487b-a39b-dcff3f37710d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:03.479918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1196/2380] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:32:10.757451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.775199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a3a47-085e-4bef-8611-4a5223da1a90 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:10.779419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:10.780164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.780472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.780744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.782800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:10.797339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:10.798726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c538e6-bb0a-4760-a059-6ab63b7bb14a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:10.810679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1185/2381] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:10.926166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee232f91-615b-4124-99e8-0d5afd0c4b37 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.926793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee232f91-615b-4124-99e8-0d5afd0c4b37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.927121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee232f91-615b-4124-99e8-0d5afd0c4b37 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.930454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1197/2382] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:10.937829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635999-02a8-42e1-ad73-c4c58720a828 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:10.938313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635999-02a8-42e1-ad73-c4c58720a828 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:10.938832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26635999-02a8-42e1-ad73-c4c58720a828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:10.962724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-26635999-02a8-42e1-ad73-c4c58720a828 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:10.963958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-26635999-02a8-42e1-ad73-c4c58720a828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:10.974436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1186/2383] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:32:10.991202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee232f91-615b-4124-99e8-0d5afd0c4b37 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:10.993333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ee232f91-615b-4124-99e8-0d5afd0c4b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:11.069498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee232f91-615b-4124-99e8-0d5afd0c4b37 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.073965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ab3c1-e411-4b7b-87bc-86758aaee8bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.074659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab3c1-e411-4b7b-87bc-86758aaee8bd service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.075069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab3c1-e411-4b7b-87bc-86758aaee8bd service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.075432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ab3c1-e411-4b7b-87bc-86758aaee8bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.088582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ab3c1-e411-4b7b-87bc-86758aaee8bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.090036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ab3c1-e411-4b7b-87bc-86758aaee8bd service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.101560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1198/2384] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:11.184391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b5c8-c372-461b-914e-3e7c6b8f7dcf None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.184633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b5c8-c372-461b-914e-3e7c6b8f7dcf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.184931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b5c8-c372-461b-914e-3e7c6b8f7dcf None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.186335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1187/2385] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:11.191552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8266e7-7077-4b4d-bbcf-12ca9b6e0f4c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.191831 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8266e7-7077-4b4d-bbcf-12ca9b6e0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.192099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8266e7-7077-4b4d-bbcf-12ca9b6e0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.210655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5b8266e7-7077-4b4d-bbcf-12ca9b6e0f4c None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:11.212070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5b8266e7-7077-4b4d-bbcf-12ca9b6e0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:11.219060 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1199/2386] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:32:11.232864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b5c8-c372-461b-914e-3e7c6b8f7dcf None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.234765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a375b5c8-c372-461b-914e-3e7c6b8f7dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:11.250460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b5c8-c372-461b-914e-3e7c6b8f7dcf None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.254322 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8419acde-7469-4ef2-9e3f-89464c45df08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.255078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419acde-7469-4ef2-9e3f-89464c45df08 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.255363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419acde-7469-4ef2-9e3f-89464c45df08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.255640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8419acde-7469-4ef2-9e3f-89464c45df08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.264330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419acde-7469-4ef2-9e3f-89464c45df08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.265706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8419acde-7469-4ef2-9e3f-89464c45df08 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.274691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1188/2387] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:11.378754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f3b5b-df48-4a57-84b8-7f2cca121a88 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.379074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f3b5b-df48-4a57-84b8-7f2cca121a88 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.379448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f3b5b-df48-4a57-84b8-7f2cca121a88 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.381551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1200/2388] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:32:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:32:11.387196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.387486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.387762 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.413848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:11.415251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:11.426418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1189/2389] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:32:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:32:11.438489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f3b5b-df48-4a57-84b8-7f2cca121a88 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.458238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f3b5b-df48-4a57-84b8-7f2cca121a88 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.464882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658d7e4-33a5-456a-95af-19d3bfd134fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.465567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658d7e4-33a5-456a-95af-19d3bfd134fb service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.465849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658d7e4-33a5-456a-95af-19d3bfd134fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.466123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658d7e4-33a5-456a-95af-19d3bfd134fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.479946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658d7e4-33a5-456a-95af-19d3bfd134fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.481428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658d7e4-33a5-456a-95af-19d3bfd134fb service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.494227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1201/2390] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:11.563445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:11.565056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:11.646748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b76dc-24cd-45a3-8481-3256289c10ca None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:11.652634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:11.653304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:11.653581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:11.653852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:11.655839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:11.668352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:11.669936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:11.680831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1190/2391] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:14.452128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.452843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.452843 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.475413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:14.477172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:14.488796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1202/2392] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:32:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:32:14.503691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:14.517193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43113ade-456c-4ddd-9ee5-62ed5136a714 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:14.523241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:14.524304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:14.524740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:14.525164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:14.535326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:14.537004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:14.547542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1191/2393] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:32:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:32:24.469242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=44 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:24.544195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=56 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:34.188560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:34.206473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9020437-a886-438b-a712-0ae056c3aef4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:34.215974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:34.217034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:34.217501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:34.217936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:34.223139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:34.253902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:34.255588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60db9c-66a6-4c58-a142-72050e2c5497 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:34.274398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1203/2394] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:35.461635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:35.475429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032fa8f2-601e-4e73-b0b8-55c95e221507 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:35.482079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:35.482768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:35.483050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:35.483325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:35.485251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:35.546381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:35.548031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:35.557611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1192/2395] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:44.269179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:45.553110 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=28 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:32:47.027247 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4103d5-41c1-4962-a40b-9a42faba310c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:47.027793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4103d5-41c1-4962-a40b-9a42faba310c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:47.028158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4103d5-41c1-4962-a40b-9a42faba310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:47.058378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ec4103d5-41c1-4962-a40b-9a42faba310c None None] MFA Rules not processed for user `28669cdf427a4012a766df06d598e54d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:47.060385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ec4103d5-41c1-4962-a40b-9a42faba310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:47.071818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1204/2396] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:32:48.509161 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:48.529777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bb04b-0fa5-40b6-bb9e-91e0b9ee26a5 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:48.538039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:48.539052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:48.539641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:48.540160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:48.543094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:48.558009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:48.560400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:48.573935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1193/2397] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:32:51.272482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252104-4cc0-4235-8729-bcec9f2e9f1b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.273100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252104-4cc0-4235-8729-bcec9f2e9f1b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.273200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb252104-4cc0-4235-8729-bcec9f2e9f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.293934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fb252104-4cc0-4235-8729-bcec9f2e9f1b None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:32:51.295315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fb252104-4cc0-4235-8729-bcec9f2e9f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:32:51.303701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1205/2398] 10.222.0.40 () {58 vars in 967 bytes} [Fri Jul 15 15:32:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:32:51.332040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:32:51.345440 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c868f-5d62-4ea0-b5ac-8ad02b938ff2 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:32:51.350930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:32:51.351712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:32:51.352005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:32:51.352284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:32:51.362413 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:32:51.364259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:32:51.374615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1194/2399] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:32:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:33:01.287630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:01.370316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=12 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:13.581416 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:13.582173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:13.582173 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:13.613111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] MFA Rules not processed for user `c1a1af21d3f94b70a830b14200619f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:33:13.614369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:13.622564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1206/2400] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:33:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:33:23.605592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:33:31.175799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:33:31.193287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c7ea4-dbe6-488e-9177-c176ab2154b2 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:33:31.201848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:33:31.203494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:33:31.203985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:33:31.204231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:33:31.206173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:33:31.220416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:33:31.222016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:33:31.235884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1195/2401] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:33:41.229777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:06.013960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:06.029957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecc600-de60-4107-ac4e-0c604b2e15a8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:06.036748 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:06.037585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:06.037963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:06.038403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:06.040736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:06.053580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:06.055287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.066598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1207/2402] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:34:06.381655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:06.394987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc7d06-9fac-4b38-a734-3b8548a84ac8 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:06.400700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:06.401780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:06.402182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:06.402576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] PATH_INFO: `/v3/users/28669cdf427a4012a766df06d598e54d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:06.403995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28669cdf427a4012a766df06d598e54d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:06.405962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.465067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28669cdf427a4012a766df06d598e54d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.481673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28669cdf427a4012a766df06d598e54d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.482576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28669cdf427a4012a766df06d598e54d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.483404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28669cdf427a4012a766df06d598e54d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.493674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28669cdf427a4012a766df06d598e54d'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.514531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1196/2403] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:34:06 2022] DELETE /identity/v3/users/28669cdf427a4012a766df06d598e54d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:06.737473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:06.756577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438040bb-6bb8-4b34-a9d1-04d272b7b2d1 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:06.837217 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:06.838099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:06.838376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:06.838687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] PATH_INFO: `/v3/projects/6fd8c6fef93d4313bb7894175242fd8c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:06.839970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd8c6fef93d4313bb7894175242fd8c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:06.842013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:06.894648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd8c6fef93d4313bb7894175242fd8c', 'request_id': 'req-e97e8485-9841-4397-99ea-f24dbb846874'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.903811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd8c6fef93d4313bb7894175242fd8c', 'request_id': 'req-e97e8485-9841-4397-99ea-f24dbb846874'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.911451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd8c6fef93d4313bb7894175242fd8c', 'request_id': 'req-e97e8485-9841-4397-99ea-f24dbb846874'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.924671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] The token cache is being invalidate because project 6fd8c6fef93d4313bb7894175242fd8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:34:06.924959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.925650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e97e8485-9841-4397-99ea-f24dbb846874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:34:06.927307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1208/2404] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:34:06 2022] DELETE /identity/v3/projects/6fd8c6fef93d4313bb7894175242fd8c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:34:16.510287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:16.921140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:22.732469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:34:22.734029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:22.933953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5471ce-8158-4bde-9cdb-cf25ef46b117 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:34:23.009986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:34:23.010596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:23.010880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:23.011196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:23.013528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:23.027262 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:34:23.028712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef81b8-4e73-4d9f-a229-46e8fa3e000f service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:34:23.043754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1197/2405] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:34:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:34:33.038892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:34:50.181894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:34:50.182762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:34:50.182919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:34:50.204909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] MFA Rules not processed for user `9a0d49b3b50b4e668e157cb2e8e47da2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:34:50.206157 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:34:50.274802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1209/2406] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:00.267406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:17.021895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:17.022767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:17.022767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:17.047704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] MFA Rules not processed for user `08050c1635084492a4b054b172884af8`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:17.051351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:17.139950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1198/2407] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:17.702009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:17.704236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:17.875027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1527ab-2b74-488a-b57a-04ff73c97d61 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:17.883946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:17.885245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:17.885789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:17.886336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:17.889806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:17.907627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:17.909806 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:17.920894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1210/2408] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:23.284057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:23.285735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:23.309007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b8c00-c1a9-496a-b902-ba4f3853b7c4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:23.315491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:23.319027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:23.319680 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:23.323095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:23.325345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:23.479233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:23.483971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:23.515084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1199/2409] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:27.426667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:27.428552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:27.448510 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3dc6d-09fd-466d-8b38-cdde9e6927c5 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:27.453922 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:27.455067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:27.455289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:27.455644 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:27.458050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:27.471976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:27.473689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:27.486285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1211/2410] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:27.493852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:27.495302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:27.510379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7f021-05e1-4870-b21b-d6078da2b9b1 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:27.518719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:27.519991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:27.521224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:27.521564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] PATH_INFO: `/v3/users/c1a1af21d3f94b70a830b14200619f87` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:27.523869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a1af21d3f94b70a830b14200619f87)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:27.525139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:27.587874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a1af21d3f94b70a830b14200619f87'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:27.598393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a1af21d3f94b70a830b14200619f87'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:27.599112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a1af21d3f94b70a830b14200619f87'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:27.599819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a1af21d3f94b70a830b14200619f87'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:27.607816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-51793cf4-acd0-4529-9af5-d42131172a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a1af21d3f94b70a830b14200619f87'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:27.609723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:27.628631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1200/2411] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:27 2022] DELETE /identity/v3/users/c1a1af21d3f94b70a830b14200619f87 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:27.836686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:27.854411 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32203417-86c2-48fa-966d-cedb32596a51 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:27.932271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:27.933496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:27.933785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:27.934062 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] PATH_INFO: `/v3/projects/1e21747a454c4ebebb5d48b5906d5e5c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:27.935587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e21747a454c4ebebb5d48b5906d5e5c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:27.938045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.004947 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e21747a454c4ebebb5d48b5906d5e5c', 'request_id': 'req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:28.016735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e21747a454c4ebebb5d48b5906d5e5c', 'request_id': 'req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:28.024081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e21747a454c4ebebb5d48b5906d5e5c', 'request_id': 'req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:28.032033 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:28.036461 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] The token cache is being invalidate because project 1e21747a454c4ebebb5d48b5906d5e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:28.036951 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:28.037906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f0354ef6-0a9c-4acb-8f7c-c3d4752a56ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:28.039915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1212/2412] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:27 2022] DELETE /identity/v3/projects/1e21747a454c4ebebb5d48b5906d5e5c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:28.047127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.047414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.047691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.123347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:28.124515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:28.194000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1201/2413] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:35:28.205385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.205936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.206410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.231084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:28.232547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:28.240898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1213/2414] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:28.245670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.256337 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c3deb-b229-45f8-a259-a16389888289 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.274666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.275731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.275731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.275731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.276483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:35:28.276793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.278699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.299781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1202/2415] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:35:28.304953 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.318737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a494f2a-98d5-433e-8fed-37142dd64e7e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.326081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.326993 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.327446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.327758 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.328635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.330643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.372297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1214/2416] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/projects => generated 391 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:28.374172 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.385619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19501497-f293-4ad4-9032-9b157b018829 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.459617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.460305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.460593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.460873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.461585 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.463571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.485579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1203/2417] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:35:28.489916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.502554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6e70-7230-448f-8af6-f840367cd3f9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.509511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.510213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.510496 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.510803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.511568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.514097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.529754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1215/2418] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:28.534020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.544702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c014a8a-513f-4052-ba01-9de48ece342e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.549508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03773caa-dc97-4050-9bcc-227f3fec1388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.550195 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773caa-dc97-4050-9bcc-227f3fec1388 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.550473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773caa-dc97-4050-9bcc-227f3fec1388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.550783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03773caa-dc97-4050-9bcc-227f3fec1388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.551517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03773caa-dc97-4050-9bcc-227f3fec1388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.553147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03773caa-dc97-4050-9bcc-227f3fec1388 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.564332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1204/2419] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:35:28.569879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.581482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ce9c7-26bf-45dd-8d12-ef7b024f44a1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.586857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.588029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.588029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.588318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] PATH_INFO: `/v3/projects/f222881a3ff8438da025591eabce8733/users/d8d06f298a8145168c9575a3263dcbc0/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.605870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f222881a3ff8438da025591eabce8733, user_id=d8d06f298a8145168c9575a3263dcbc0, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.608687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.624020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1216/2420] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:28 2022] PUT /identity/v3/projects/f222881a3ff8438da025591eabce8733/users/d8d06f298a8145168c9575a3263dcbc0/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:28.634893 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.635136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.635357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.666352 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] MFA Rules not processed for user `d8d06f298a8145168c9575a3263dcbc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:28.667731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:28.675364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:28.759332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1205/2421] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:28.765046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.844598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e193b-9230-468f-a7bb-b9356448d555 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.850024 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.850727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.851007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.851280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.851975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.853929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:28.888620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1217/2422] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/projects => generated 391 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:28.892951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:28.910693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b834ab4-f956-41f7-8f25-36d5207f9790 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:28.977733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:28.978446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:28.978857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:28.979281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:28.980330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:28.982948 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.008014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1206/2423] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:28 2022] POST /identity/v3/users => generated 302 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:29.012343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:29.027606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d38cdd-2842-43d6-beeb-f6a2d1514add admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:29.032622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:29.033293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.033783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.033913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.034594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:29.036271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.050366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1218/2424] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:29.054999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:29.067615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2237-75f0-4058-bace-aa54713edad7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:29.074675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:29.075720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.076204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.076641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] PATH_INFO: `/v3/projects/d730477a21f84306bd9502af03df1fd0/users/84b8c787d9fa4914a3ad1c2914d425db/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.102650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d730477a21f84306bd9502af03df1fd0, user_id=84b8c787d9fa4914a3ad1c2914d425db, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:29.105920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.122876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1207/2425] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:29 2022] PUT /identity/v3/projects/d730477a21f84306bd9502af03df1fd0/users/84b8c787d9fa4914a3ad1c2914d425db/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:29.127506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:29.139348 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924304de-fc1a-4b8e-81ab-b6e0c12d12f4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:29.224547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1f5f-acfb-4dab-a6a1-63e361caeb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:29.225267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1f5f-acfb-4dab-a6a1-63e361caeb85 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.225573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1f5f-acfb-4dab-a6a1-63e361caeb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.226580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1f5f-acfb-4dab-a6a1-63e361caeb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.226580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1f5f-acfb-4dab-a6a1-63e361caeb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:29.228357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1f5f-acfb-4dab-a6a1-63e361caeb85 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.242142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1219/2426] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:35:29.247823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:29.260668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702095eb-d149-4d43-b322-7e3c022b564d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:29.266281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:29.267241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.267521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.267798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] PATH_INFO: `/v3/projects/d730477a21f84306bd9502af03df1fd0/users/84b8c787d9fa4914a3ad1c2914d425db/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.270056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d730477a21f84306bd9502af03df1fd0, user_id=84b8c787d9fa4914a3ad1c2914d425db, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:29.272676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.284908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1208/2427] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:29 2022] PUT /identity/v3/projects/d730477a21f84306bd9502af03df1fd0/users/84b8c787d9fa4914a3ad1c2914d425db/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:29.295192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.295607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.295765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.309506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:29.335492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] MFA Rules not processed for user `84b8c787d9fa4914a3ad1c2914d425db`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:29.337557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:29.463596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1220/2428] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:29.475081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:29.477264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:29.571652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:29.591444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68c789-e014-40be-a60b-e39231e8851a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:29.595801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:29.596672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.596955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.597236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.609772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:29.611704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.625095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1209/2429] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:29.682207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:29.684427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:29.699029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ab023-dc44-4b95-98bc-ddc8f11d3142 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:29.703616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89637a-cf0a-4ffd-aa1f-922bf9aaeca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:29.704405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89637a-cf0a-4ffd-aa1f-922bf9aaeca6 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:29.704686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89637a-cf0a-4ffd-aa1f-922bf9aaeca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:29.705095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89637a-cf0a-4ffd-aa1f-922bf9aaeca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:29.782927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89637a-cf0a-4ffd-aa1f-922bf9aaeca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:29.785307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89637a-cf0a-4ffd-aa1f-922bf9aaeca6 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:29.800336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1221/2430] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:33.505278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:33.612045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41e116-6154-4262-b651-511e48d58116 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:33.617974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:33.619583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:33.620012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:33.620301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] PATH_INFO: `/v3/users/d8d06f298a8145168c9575a3263dcbc0` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:33.621742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8d06f298a8145168c9575a3263dcbc0)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:33.623784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:33.700051 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8d06f298a8145168c9575a3263dcbc0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:33.717870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8d06f298a8145168c9575a3263dcbc0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:33.719977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8d06f298a8145168c9575a3263dcbc0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:33.720817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8d06f298a8145168c9575a3263dcbc0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:33.730269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8d06f298a8145168c9575a3263dcbc0'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:33.759842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1210/2431] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:33 2022] DELETE /identity/v3/users/d8d06f298a8145168c9575a3263dcbc0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:33.765659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:33.766508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:33.767593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:33.789096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:33.790351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:33.878935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1222/2432] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:33.890101 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:33.892484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:33.997377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd092-388e-44e7-988c-a8419fd2472a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:34.003487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:34.003814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:34.004033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:34.004398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:34.017293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:34.019105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:34.035765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1211/2433] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:35:34.417822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:34.421055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:34.441234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455fcaf-b764-4b32-8cc3-1a6eccf14fe3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:34.448944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:34.450402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:34.450981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:34.451431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] PATH_INFO: `/v3/projects/f222881a3ff8438da025591eabce8733` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:34.453527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f222881a3ff8438da025591eabce8733)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:34.456317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:34.494225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:34.513113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f222881a3ff8438da025591eabce8733', 'request_id': 'req-5083ce19-e2c8-48b5-a515-8053fd7b34fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.523790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f222881a3ff8438da025591eabce8733', 'request_id': 'req-5083ce19-e2c8-48b5-a515-8053fd7b34fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.532151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f222881a3ff8438da025591eabce8733', 'request_id': 'req-5083ce19-e2c8-48b5-a515-8053fd7b34fc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.544677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] The token cache is being invalidate because project f222881a3ff8438da025591eabce8733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:34.544974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.545663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.548152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1223/2434] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:34 2022] DELETE /identity/v3/projects/f222881a3ff8438da025591eabce8733 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:35:34.552069 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:34.558575 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:34.563383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb1e7c-c507-4c6f-8ae1-42e347ccbc9a service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:34.631751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:34.632655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:34.632942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:34.633221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] PATH_INFO: `/v3/users/84b8c787d9fa4914a3ad1c2914d425db` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:34.634446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b8c787d9fa4914a3ad1c2914d425db)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:34.636591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:34.694374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b8c787d9fa4914a3ad1c2914d425db'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.703673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b8c787d9fa4914a3ad1c2914d425db'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.704547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b8c787d9fa4914a3ad1c2914d425db'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.705292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b8c787d9fa4914a3ad1c2914d425db'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.713580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-aa79713a-2acb-4e1e-aa68-c3e8ba9019b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b8c787d9fa4914a3ad1c2914d425db'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:34.734419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1212/2435] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:34 2022] DELETE /identity/v3/users/84b8c787d9fa4914a3ad1c2914d425db => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:35.068462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.070636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:35.085967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083ce19-e2c8-48b5-a515-8053fd7b34fc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.149534 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.150413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.150727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.151035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] PATH_INFO: `/v3/projects/d730477a21f84306bd9502af03df1fd0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.152295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d730477a21f84306bd9502af03df1fd0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.154218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.207676 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd730477a21f84306bd9502af03df1fd0', 'request_id': 'req-3b04a655-b8e0-42c2-a928-009853ae1b18'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:35.216560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd730477a21f84306bd9502af03df1fd0', 'request_id': 'req-3b04a655-b8e0-42c2-a928-009853ae1b18'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:35.223778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd730477a21f84306bd9502af03df1fd0', 'request_id': 'req-3b04a655-b8e0-42c2-a928-009853ae1b18'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:35.235762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] The token cache is being invalidate because project d730477a21f84306bd9502af03df1fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:35.236038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:35.236939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3b04a655-b8e0-42c2-a928-009853ae1b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:35.239144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1224/2436] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:35 2022] DELETE /identity/v3/projects/d730477a21f84306bd9502af03df1fd0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:35.244137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.244428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.244706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.263982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:35.265144 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:35.327222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1213/2437] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:35.338675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.338983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.339275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.358470 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:35.359713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:35.368239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1225/2438] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:35.372717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.383977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135ba2e-ad1a-4220-8d9c-1ea99d6bcb21 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.389498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.390311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.390697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.391056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.392035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:35:35.392407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.394108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.410208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1214/2439] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:35:35.414601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.425900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88e17b-c117-4b06-94ce-f161f988c865 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.431095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.431767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.432049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.432360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.433058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.435041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.464124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1226/2440] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:35.469179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.485963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4494-914a-49c3-beda-8f73b0d0692a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.495350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:35.550049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.550698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.550979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.551245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.551944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.553867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.574962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1215/2441] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:35.579397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.590206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b1584-f70a-4c59-af70-d2fe365cd966 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.595010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.595571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.595849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.596120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.596848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.598495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.610948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1227/2442] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:35.615514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.630612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc8bdf-409d-4838-9400-f9fe3e8816dc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.636171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac502849-c653-4870-88fe-fcf8e3b4894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.636870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502849-c653-4870-88fe-fcf8e3b4894e admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.637160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502849-c653-4870-88fe-fcf8e3b4894e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.637436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac502849-c653-4870-88fe-fcf8e3b4894e admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.638170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502849-c653-4870-88fe-fcf8e3b4894e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.639842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac502849-c653-4870-88fe-fcf8e3b4894e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.652148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1216/2443] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:35.656528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.664012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:35.668149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd76baf-f9b9-49d5-af8d-3f3d45286778 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.675933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.676859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.677148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.677435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] PATH_INFO: `/v3/projects/cfaa2e075d3e477ca07fe054bf7451fc/users/11087830d7474d5bb1e64f9f134329f2/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.695222 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfaa2e075d3e477ca07fe054bf7451fc, user_id=11087830d7474d5bb1e64f9f134329f2, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.697643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.712724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1228/2444] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:35 2022] PUT /identity/v3/projects/cfaa2e075d3e477ca07fe054bf7451fc/users/11087830d7474d5bb1e64f9f134329f2/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:35.723586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.723877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.724154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.754777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] MFA Rules not processed for user `11087830d7474d5bb1e64f9f134329f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:35.756111 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:35.848071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1217/2445] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:35.852841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.928907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8191c4-0ec7-4dd0-a472-f3f570be248a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:35.934136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:35.934911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:35.935099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:35.935399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:35.936094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:35.938037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:35.973626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1229/2446] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:35.978536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:35.995599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b99ff9-e7fb-4204-9150-6253455938ae None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:36.065699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:36.066452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.066804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.067092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.067819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:36.069972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:36.097429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1218/2447] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:35 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:36.101541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:36.113057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487323df-3bcd-419f-a193-b28a5ca5c364 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:36.119031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:36.119802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.120079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.120364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.121112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:36.122847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:36.137345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1230/2448] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:36.141865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:36.154001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b143081-12e5-4693-a322-febff0aae92a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:36.167094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:36.169617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.169617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.169617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] PATH_INFO: `/v3/projects/734fad99fa8a4e5fa1391009b2d6db90/users/e35df0c7d449410688a42b5d9cbd0a85/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.172922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:36.190089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] RBAC: Authorizing `identity:create_grant(project_id=734fad99fa8a4e5fa1391009b2d6db90, user_id=e35df0c7d449410688a42b5d9cbd0a85, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:36.192778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:36.208716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1219/2449] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:36 2022] PUT /identity/v3/projects/734fad99fa8a4e5fa1391009b2d6db90/users/e35df0c7d449410688a42b5d9cbd0a85/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:36.212876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:36.226916 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa64b5-8867-474f-8638-35f12c88d490 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:36.297604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602bdc-2fda-45da-9ead-f7e76c3fbb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:36.298407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602bdc-2fda-45da-9ead-f7e76c3fbb48 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.299111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602bdc-2fda-45da-9ead-f7e76c3fbb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.299111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602bdc-2fda-45da-9ead-f7e76c3fbb48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.299742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602bdc-2fda-45da-9ead-f7e76c3fbb48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:36.301382 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602bdc-2fda-45da-9ead-f7e76c3fbb48 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:36.312773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1231/2450] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:35:36.318067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:36.329131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a5f71-2bfc-4caf-8aa9-d26c025f496f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:36.336062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:36.336972 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.337261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.337541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] PATH_INFO: `/v3/projects/734fad99fa8a4e5fa1391009b2d6db90/users/e35df0c7d449410688a42b5d9cbd0a85/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.339686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734fad99fa8a4e5fa1391009b2d6db90, user_id=e35df0c7d449410688a42b5d9cbd0a85, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:36.342347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:36.355657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1220/2451] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:36 2022] PUT /identity/v3/projects/734fad99fa8a4e5fa1391009b2d6db90/users/e35df0c7d449410688a42b5d9cbd0a85/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:36.368794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.369210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.369568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.412917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] MFA Rules not processed for user `e35df0c7d449410688a42b5d9cbd0a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:36.412917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:36.460970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:36.508875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1232/2452] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:36.524499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:36.526751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:36.610456 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c20abb-ee35-47ef-8e9e-abc198ac90f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:36.616526 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993f214-146c-4181-adc5-167ce340113a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:36.617579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993f214-146c-4181-adc5-167ce340113a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:36.618042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993f214-146c-4181-adc5-167ce340113a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:36.618510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993f214-146c-4181-adc5-167ce340113a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:36.632733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993f214-146c-4181-adc5-167ce340113a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:36.634450 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7993f214-146c-4181-adc5-167ce340113a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:36.646296 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1221/2453] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:37.324422 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:37.400310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9600c4-1e36-4c6c-a559-4dd81be94abf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:37.405384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:37.406284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:37.406745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:37.406929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] PATH_INFO: `/v3/users/11087830d7474d5bb1e64f9f134329f2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:37.408262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11087830d7474d5bb1e64f9f134329f2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:37.410232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:37.469419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11087830d7474d5bb1e64f9f134329f2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:37.477577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11087830d7474d5bb1e64f9f134329f2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:37.486667 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11087830d7474d5bb1e64f9f134329f2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:37.487385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11087830d7474d5bb1e64f9f134329f2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:37.487996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11087830d7474d5bb1e64f9f134329f2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:37.510666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1233/2454] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:37 2022] DELETE /identity/v3/users/11087830d7474d5bb1e64f9f134329f2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:37.516804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:37.517122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:37.517354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:37.537278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:37.538544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:37.610975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1222/2455] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:37.621369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:37.623058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:37.693530 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ad072-569d-4dee-8c34-8e393310cc21 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:37.700022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:37.700693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:37.700973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:37.701243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:37.714166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:37.715713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:37.720139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:37.728824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1234/2456] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:38.083401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:38.085305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:38.102190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c1f67-6e89-4fdd-9255-364e578722f4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:38.109801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:38.111072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:38.112013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:38.112013 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] PATH_INFO: `/v3/projects/cfaa2e075d3e477ca07fe054bf7451fc` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:38.113562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfaa2e075d3e477ca07fe054bf7451fc)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:38.115833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:38.180188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfaa2e075d3e477ca07fe054bf7451fc', 'request_id': 'req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.187171 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfaa2e075d3e477ca07fe054bf7451fc', 'request_id': 'req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.194359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfaa2e075d3e477ca07fe054bf7451fc', 'request_id': 'req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.205950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:38.213173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] The token cache is being invalidate because project cfaa2e075d3e477ca07fe054bf7451fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:38.213666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.214896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.217929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1223/2457] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:38 2022] DELETE /identity/v3/projects/cfaa2e075d3e477ca07fe054bf7451fc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:38.222761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:38.233904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a3162-bda9-4b3c-aa6f-85b2ddffca68 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:38.304349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:38.305258 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:38.305541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:38.305814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] PATH_INFO: `/v3/users/e35df0c7d449410688a42b5d9cbd0a85` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:38.307123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35df0c7d449410688a42b5d9cbd0a85)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:38.309136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:38.369264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35df0c7d449410688a42b5d9cbd0a85'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.378688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35df0c7d449410688a42b5d9cbd0a85'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.389872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35df0c7d449410688a42b5d9cbd0a85'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.389872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35df0c7d449410688a42b5d9cbd0a85'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.390275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4052c97-b151-446e-aa11-53be3a0ee33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35df0c7d449410688a42b5d9cbd0a85'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.411489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1235/2458] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:38 2022] DELETE /identity/v3/users/e35df0c7d449410688a42b5d9cbd0a85 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:38.758276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:38.759835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:38.788383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe92a7b-c8fe-4b4d-b606-bf3e74e35fd5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:38.876050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:38.876319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:38.876576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:38.876848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] PATH_INFO: `/v3/projects/734fad99fa8a4e5fa1391009b2d6db90` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:38.878565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734fad99fa8a4e5fa1391009b2d6db90)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:38.880825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:38.950928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734fad99fa8a4e5fa1391009b2d6db90', 'request_id': 'req-15414afc-a1b5-4078-9761-34e788f6dfc8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.960708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734fad99fa8a4e5fa1391009b2d6db90', 'request_id': 'req-15414afc-a1b5-4078-9761-34e788f6dfc8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.967203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734fad99fa8a4e5fa1391009b2d6db90', 'request_id': 'req-15414afc-a1b5-4078-9761-34e788f6dfc8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.994677 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] The token cache is being invalidate because project 734fad99fa8a4e5fa1391009b2d6db90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:38.994962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.995921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-15414afc-a1b5-4078-9761-34e788f6dfc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:38.997651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1224/2459] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:38 2022] DELETE /identity/v3/projects/734fad99fa8a4e5fa1391009b2d6db90 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:39.003526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebde15-1b08-48df-918d-b219957de84d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.003802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebde15-1b08-48df-918d-b219957de84d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.004009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebde15-1b08-48df-918d-b219957de84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.024489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-adebde15-1b08-48df-918d-b219957de84d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:39.025736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-adebde15-1b08-48df-918d-b219957de84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:39.100124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1236/2460] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:39.108208 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.108594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.108877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.135689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:39.137615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:39.148131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1225/2461] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:35:39.153130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebde15-1b08-48df-918d-b219957de84d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.164134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebde15-1b08-48df-918d-b219957de84d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.169474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.170165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.170421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.170886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.171450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:35:39.173086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.173441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.190780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1237/2462] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:35:39.195949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.206738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716d08c-4fe0-46f3-ac09-c04ae431650f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.211986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.212777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.213080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.213354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.214063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.216272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.249486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1226/2463] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:35:39.254158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.261273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:39.265847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad6a38-13f9-4861-b667-6f8ed1837a1a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.356111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.356724 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.357006 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.357281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.357995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.360042 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.389474 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1238/2464] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:39.397068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.409570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c1396-01ed-4410-b3a2-0f8dc9181380 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.418334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.419085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.419366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.419672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.420421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.422419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.435565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1227/2465] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:39.440623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.457107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f781145-55e1-49e4-8497-093a03d280ac admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.466549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9041b9-18b3-4261-b3c7-d12b32353d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.467236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9041b9-18b3-4261-b3c7-d12b32353d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.467515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9041b9-18b3-4261-b3c7-d12b32353d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.467786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9041b9-18b3-4261-b3c7-d12b32353d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.468516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9041b9-18b3-4261-b3c7-d12b32353d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.470178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9041b9-18b3-4261-b3c7-d12b32353d58 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.482753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1239/2466] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:39.487578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.500930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f174c-651a-4b44-9fdf-836c7de4687d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.506042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.507004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.507283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.507554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] PATH_INFO: `/v3/projects/8da0ed4d3be04a2f85f3f0ed4816a3f9/users/667b3fa43b3149ef91edfe152d28ce13/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.524026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da0ed4d3be04a2f85f3f0ed4816a3f9, user_id=667b3fa43b3149ef91edfe152d28ce13, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.526725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.546755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1228/2467] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:39 2022] PUT /identity/v3/projects/8da0ed4d3be04a2f85f3f0ed4816a3f9/users/667b3fa43b3149ef91edfe152d28ce13/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:39.558020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.558309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.558642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.592367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] MFA Rules not processed for user `667b3fa43b3149ef91edfe152d28ce13`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:39.594915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:39.671936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1240/2468] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:39.676649 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.710128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:39.748552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16160c-9830-4d48-9a97-655905c54f90 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.757998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.758711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.759304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.759304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.759956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.761894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.799124 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1229/2469] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:39.806800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.819514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75de0b-6174-41e2-8ee8-681ece9e48ab None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.883438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.884128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.884418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.884693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.885398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.887393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.917318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1241/2470] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:39 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:35:39.922041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.934100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0651ba-9452-40c5-8b4d-04e61237bf39 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.940792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.941834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.942126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.942397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:39.943175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:39.944858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:39.961634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1230/2471] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:39.967840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:39.973668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:39.980019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77d187-9636-495c-8d36-c84328a9389a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:39.985786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:39.986731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:39.987009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:39.987309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] PATH_INFO: `/v3/projects/05670cf3aa0d4a39a015cd5ea0722c5d/users/a3e30fb870e64dd498c71eb8ad8d1af7/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:40.005161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05670cf3aa0d4a39a015cd5ea0722c5d, user_id=a3e30fb870e64dd498c71eb8ad8d1af7, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:40.009445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:40.025127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1242/2472] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:39 2022] PUT /identity/v3/projects/05670cf3aa0d4a39a015cd5ea0722c5d/users/a3e30fb870e64dd498c71eb8ad8d1af7/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:40.030817 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:40.044203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa943d29-bd76-4c0f-8b9a-4d0d89b5e418 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:40.104989 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca37f7-efa3-42e8-8be5-e3301989ba0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:40.105675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca37f7-efa3-42e8-8be5-e3301989ba0d admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:40.105955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca37f7-efa3-42e8-8be5-e3301989ba0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:40.106204 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca37f7-efa3-42e8-8be5-e3301989ba0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:40.106987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca37f7-efa3-42e8-8be5-e3301989ba0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:40.108625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca37f7-efa3-42e8-8be5-e3301989ba0d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:40.123119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1231/2473] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:35:40.127088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:40.141386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b73fda-0bbb-4250-b1af-22644f238860 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:40.146979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:40.147894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:40.148172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:40.148443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] PATH_INFO: `/v3/projects/05670cf3aa0d4a39a015cd5ea0722c5d/users/a3e30fb870e64dd498c71eb8ad8d1af7/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:40.150910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05670cf3aa0d4a39a015cd5ea0722c5d, user_id=a3e30fb870e64dd498c71eb8ad8d1af7, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:40.155134 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:40.169324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1243/2474] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:40 2022] PUT /identity/v3/projects/05670cf3aa0d4a39a015cd5ea0722c5d/users/a3e30fb870e64dd498c71eb8ad8d1af7/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:40.182572 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:40.182769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:40.183057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:40.212528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] MFA Rules not processed for user `a3e30fb870e64dd498c71eb8ad8d1af7`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:40.213788 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:40.415048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1232/2475] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:40.426019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:40.427862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:40.486084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a50f2-5446-4627-84b7-e25e75bc2a21 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:40.490124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:40.490800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:40.491081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:40.491355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:40.553863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:40.555361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:40.564788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1244/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:44.834026 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:44.835746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:44.848369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f9a97-8575-4d20-999e-85b2758960a4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:44.852896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c40fc-6e74-4904-8f06-39c0b9946eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:44.853538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c40fc-6e74-4904-8f06-39c0b9946eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:44.853822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c40fc-6e74-4904-8f06-39c0b9946eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:44.854059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c40fc-6e74-4904-8f06-39c0b9946eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:44.863858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c40fc-6e74-4904-8f06-39c0b9946eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:44.865574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c40fc-6e74-4904-8f06-39c0b9946eb5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:44.875191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1233/2477] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:35:49.829289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:49.897386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0aaae4-e06d-4c63-968d-d59122faea98 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:49.903339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:49.904218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:49.904493 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:49.904759 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] PATH_INFO: `/v3/users/667b3fa43b3149ef91edfe152d28ce13` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:49.905957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=667b3fa43b3149ef91edfe152d28ce13)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:49.907934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:49.930898 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:49.960342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '667b3fa43b3149ef91edfe152d28ce13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:49.969194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '667b3fa43b3149ef91edfe152d28ce13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:49.978374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '667b3fa43b3149ef91edfe152d28ce13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:49.979053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '667b3fa43b3149ef91edfe152d28ce13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:49.979684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '667b3fa43b3149ef91edfe152d28ce13'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:49.997345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1245/2478] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:49 2022] DELETE /identity/v3/users/667b3fa43b3149ef91edfe152d28ce13 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:50.002183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:50.002714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:50.002841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:50.015476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:50.021159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:50.023088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:50.090278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1234/2479] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:50.100336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:50.102570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:50.165630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e091c3-e7ee-4515-a7b5-5b2a4dfc7541 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:50.169751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:50.170398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:50.170711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:50.170986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:50.180441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:50.181959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:50.191622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1246/2480] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:50.474404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:50.475960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:50.494588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe092f-cbf1-407b-ae9e-182743112156 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:50.499499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:50.500404 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:50.500668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:50.500957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] PATH_INFO: `/v3/projects/8da0ed4d3be04a2f85f3f0ed4816a3f9` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:50.502191 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da0ed4d3be04a2f85f3f0ed4816a3f9)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:50.504229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:50.556922 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da0ed4d3be04a2f85f3f0ed4816a3f9', 'request_id': 'req-6db4cbef-fbd5-4a29-a86b-db39b890fe59'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.563175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da0ed4d3be04a2f85f3f0ed4816a3f9', 'request_id': 'req-6db4cbef-fbd5-4a29-a86b-db39b890fe59'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.568957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da0ed4d3be04a2f85f3f0ed4816a3f9', 'request_id': 'req-6db4cbef-fbd5-4a29-a86b-db39b890fe59'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.584607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] The token cache is being invalidate because project 8da0ed4d3be04a2f85f3f0ed4816a3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:50.584892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.585531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.587742 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1235/2481] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:50 2022] DELETE /identity/v3/projects/8da0ed4d3be04a2f85f3f0ed4816a3f9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:50.591810 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:50.603064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd70f7-9467-4896-ae64-180efc734117 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:50.673119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:50.674393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:50.674903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:50.675220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] PATH_INFO: `/v3/users/a3e30fb870e64dd498c71eb8ad8d1af7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:50.676595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e30fb870e64dd498c71eb8ad8d1af7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:50.679473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:50.729137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:50.731902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e30fb870e64dd498c71eb8ad8d1af7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.741890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e30fb870e64dd498c71eb8ad8d1af7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.750407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e30fb870e64dd498c71eb8ad8d1af7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.751082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e30fb870e64dd498c71eb8ad8d1af7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.751718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0d6a99a9-8054-4584-9553-16265c7e910c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e30fb870e64dd498c71eb8ad8d1af7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:50.772051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1247/2482] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:50 2022] DELETE /identity/v3/users/a3e30fb870e64dd498c71eb8ad8d1af7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:51.076369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.077906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:51.091712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4cbef-fbd5-4a29-a86b-db39b890fe59 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.152420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.153242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.153516 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.153780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] PATH_INFO: `/v3/projects/05670cf3aa0d4a39a015cd5ea0722c5d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.155019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05670cf3aa0d4a39a015cd5ea0722c5d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.156871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.203131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:51.218273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05670cf3aa0d4a39a015cd5ea0722c5d', 'request_id': 'req-cc593135-136c-48e1-a9fc-f5388173cf60'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:51.224986 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05670cf3aa0d4a39a015cd5ea0722c5d', 'request_id': 'req-cc593135-136c-48e1-a9fc-f5388173cf60'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:51.231994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05670cf3aa0d4a39a015cd5ea0722c5d', 'request_id': 'req-cc593135-136c-48e1-a9fc-f5388173cf60'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:51.246250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] The token cache is being invalidate because project 05670cf3aa0d4a39a015cd5ea0722c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:51.246535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:51.247317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cc593135-136c-48e1-a9fc-f5388173cf60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:51.248944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1236/2483] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:51 2022] DELETE /identity/v3/projects/05670cf3aa0d4a39a015cd5ea0722c5d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:51.255188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.255757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.255757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.278297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:51.279584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:51.341822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1248/2484] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:35:51.350544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.350849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.351129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.371024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:51.372258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:51.381465 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1237/2485] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:51.386362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.397114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378463b4-c3c5-46aa-9221-3dd9b9078f08 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.404360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.405270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.405652 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.406038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.407233 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:35:51.407673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.410196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.430908 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1249/2486] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:35:51.436479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.447014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efecd5-86f7-49db-9d5a-9235c08b59c1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.452979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.453645 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.453925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.454198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.455061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.456974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.485330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1238/2487] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:51 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:51.490374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.507208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334b03d-0379-4ddf-9043-44226fa43f2d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.566871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.567552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.567834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.568110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.568804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.570871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.591191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1250/2488] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:51 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:51.596825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.607524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc6caf-3899-409d-ab7f-7889a5207c5d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.612387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.613031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.613574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.614359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.615738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.617376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.628509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1239/2489] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:35:51.635335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.648511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023a3ae-e808-439e-9d6d-42024f7f0a09 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.657795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefcc86-d7ba-4b3a-9810-253f3f6ee6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.658406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefcc86-d7ba-4b3a-9810-253f3f6ee6cd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.658718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefcc86-d7ba-4b3a-9810-253f3f6ee6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.658996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefcc86-d7ba-4b3a-9810-253f3f6ee6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.659732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefcc86-d7ba-4b3a-9810-253f3f6ee6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.664101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefcc86-d7ba-4b3a-9810-253f3f6ee6cd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.677478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1251/2490] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:51.684083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.698867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31184594-3eb6-428b-bd20-87ac027b93af admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.705188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.706092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.706374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.706685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] PATH_INFO: `/v3/projects/2d05d74a2b144950af21039ad2dd2203/users/1cc21b5941c943579c9f2ee5cf941c10/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.723899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d05d74a2b144950af21039ad2dd2203, user_id=1cc21b5941c943579c9f2ee5cf941c10, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.727224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.742578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1240/2491] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:51 2022] PUT /identity/v3/projects/2d05d74a2b144950af21039ad2dd2203/users/1cc21b5941c943579c9f2ee5cf941c10/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:51.754178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.754501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.754826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.783172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] MFA Rules not processed for user `1cc21b5941c943579c9f2ee5cf941c10`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:51.784389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:51.797480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:51.852213 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1252/2492] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:51.864062 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:51.866150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:51.943120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed771bad-5411-4131-ab4e-720ba75bfcb8 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:51.947476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83560e08-8f38-44dc-a1e0-a578e436c3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:51.948118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560e08-8f38-44dc-a1e0-a578e436c3bf service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:51.948399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560e08-8f38-44dc-a1e0-a578e436c3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:51.948671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83560e08-8f38-44dc-a1e0-a578e436c3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:51.958418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83560e08-8f38-44dc-a1e0-a578e436c3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:51.959884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83560e08-8f38-44dc-a1e0-a578e436c3bf service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:51.969021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1241/2493] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:55.123649 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:55.196093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae848e9e-7615-45a5-967c-64dbb365f933 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:55.201379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:55.202265 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:55.202547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:55.202851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] PATH_INFO: `/v3/users/1cc21b5941c943579c9f2ee5cf941c10` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:55.204056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc21b5941c943579c9f2ee5cf941c10)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:55.206075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:55.260997 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc21b5941c943579c9f2ee5cf941c10'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.268671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc21b5941c943579c9f2ee5cf941c10'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.276209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc21b5941c943579c9f2ee5cf941c10'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.276822 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc21b5941c943579c9f2ee5cf941c10'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.277448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc21b5941c943579c9f2ee5cf941c10'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.295160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1253/2494] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:35:55 2022] DELETE /identity/v3/users/1cc21b5941c943579c9f2ee5cf941c10 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:55.300587 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:55.301309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:55.301309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:55.323108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:55.324556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:55.378134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:55.390284 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1242/2495] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:35:55.400249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:55.401944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:55.444029 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:55.464089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8238d23-b59c-4463-807f-dd8fc659765d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:55.468322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29070439-9735-4005-8de4-a0dd547f6027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:55.468977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070439-9735-4005-8de4-a0dd547f6027 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:55.469260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070439-9735-4005-8de4-a0dd547f6027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:55.469535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29070439-9735-4005-8de4-a0dd547f6027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:55.480413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29070439-9735-4005-8de4-a0dd547f6027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:55.481867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29070439-9735-4005-8de4-a0dd547f6027 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:55.492251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1254/2496] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:55.777217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:55.778914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:55.795958 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65688543-9cf7-4f2a-bcf2-c1ffad778620 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:55.804145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:55.805130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:55.805480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:55.805681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] PATH_INFO: `/v3/projects/2d05d74a2b144950af21039ad2dd2203` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:55.807502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d05d74a2b144950af21039ad2dd2203)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:55.809321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:55.857925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d05d74a2b144950af21039ad2dd2203', 'request_id': 'req-58bfb06d-6445-456a-8501-e3e1c0642451'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.864706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d05d74a2b144950af21039ad2dd2203', 'request_id': 'req-58bfb06d-6445-456a-8501-e3e1c0642451'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.871192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d05d74a2b144950af21039ad2dd2203', 'request_id': 'req-58bfb06d-6445-456a-8501-e3e1c0642451'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.884931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] The token cache is being invalidate because project 2d05d74a2b144950af21039ad2dd2203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:35:55.885547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.886180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-58bfb06d-6445-456a-8501-e3e1c0642451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:35:55.887756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1243/2497] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:35:55 2022] DELETE /identity/v3/projects/2d05d74a2b144950af21039ad2dd2203 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:55.894028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:55.894320 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:55.894633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:55.912418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:55.913653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:55.974377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1255/2498] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:55.982251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:55.982581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:55.982874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.002398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:56.003655 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:56.012416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1244/2499] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:56.017307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.027367 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d87cd-54ab-4bce-8510-e3fe81a34b54 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.032334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.032992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.033274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.033560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.034292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:35:56.034628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.036293 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.048428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1256/2500] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:35:56.052913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.062552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c5378-1c89-4bdb-a03b-b61dfd452ec7 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.067531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.068290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.068678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.069046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.069829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.071885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.098449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1245/2501] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:35:56 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:56.102927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.117663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2faa548-6160-42e5-acbb-5dcc8ed1bad9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.183802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.184599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.184880 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.185206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.185992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.188078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.215321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1257/2502] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:35:56 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:35:56.219995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.234503 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde411c-8e66-493a-9ec5-6fa7f6762504 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.240892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.241878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.242305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.242833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.244050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.246711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.260855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1246/2503] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:56.266103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.281211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56268236-a4a3-4d5a-947f-f5b9dc3b6c39 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.286446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076794a-7253-471c-aca6-b4d3e883b687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.287159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076794a-7253-471c-aca6-b4d3e883b687 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.287441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076794a-7253-471c-aca6-b4d3e883b687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.287715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076794a-7253-471c-aca6-b4d3e883b687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.288477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076794a-7253-471c-aca6-b4d3e883b687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.290082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076794a-7253-471c-aca6-b4d3e883b687 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.301488 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1258/2504] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:35:56.307017 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.322339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b455f3e-97cd-4d91-8720-4ca622701bc4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.333108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.334291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.334609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.334890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] PATH_INFO: `/v3/projects/af8ddde0deb24e7a9222ec5da477018e/users/d590d4ec416b4e59b1133f3abf86fc7e/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.352472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8ddde0deb24e7a9222ec5da477018e, user_id=d590d4ec416b4e59b1133f3abf86fc7e, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.357620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.380355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1247/2505] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:35:56 2022] PUT /identity/v3/projects/af8ddde0deb24e7a9222ec5da477018e/users/d590d4ec416b4e59b1133f3abf86fc7e/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:35:56.391744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19197ca-3cb1-46db-af19-96707f53e799 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.392072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19197ca-3cb1-46db-af19-96707f53e799 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.392355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19197ca-3cb1-46db-af19-96707f53e799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.424084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d19197ca-3cb1-46db-af19-96707f53e799 None None] MFA Rules not processed for user `d590d4ec416b4e59b1133f3abf86fc7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:56.425369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d19197ca-3cb1-46db-af19-96707f53e799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:56.473254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:35:56.516282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1259/2506] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:35:56.530055 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:35:56.532023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:56.603549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3189e03-7f65-4ca6-a576-77608cc143ee admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:35:56.608120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:35:56.608773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:56.609052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:56.609331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:56.618461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:35:56.620029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:35:56.629909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1248/2507] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:35:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:35:58.030131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0569f3-4a7b-47a8-b665-eaaa80dd63fc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:35:58.030787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0569f3-4a7b-47a8-b665-eaaa80dd63fc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:35:58.030787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0569f3-4a7b-47a8-b665-eaaa80dd63fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:35:58.060077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c0569f3-4a7b-47a8-b665-eaaa80dd63fc None None] MFA Rules not processed for user `d590d4ec416b4e59b1133f3abf86fc7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:35:58.061329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c0569f3-4a7b-47a8-b665-eaaa80dd63fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:35:58.069658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1260/2508] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:06.625635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:08.049570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=15 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:13.045207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:13.047217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:13.110830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b7ea5-fbb0-4476-894c-976489afa6d2 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:13.115036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:13.115707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:13.115966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:13.116236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:13.118251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:13.226808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:13.228253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:13.259257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1249/2509] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:36:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:36:14.484425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e83ab-44e3-488a-8589-103bf5c56ae6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:14.484954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e83ab-44e3-488a-8589-103bf5c56ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:14.485304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e83ab-44e3-488a-8589-103bf5c56ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:14.529934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-470e83ab-44e3-488a-8589-103bf5c56ae6 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:14.531662 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-470e83ab-44e3-488a-8589-103bf5c56ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:14.591088 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1261/2510] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:36:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:14.656177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:14.658320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:14.732310 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e2f94-cd01-4a22-9556-985e7d39d0de service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:14.737664 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:14.738336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:14.738640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:14.738913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:14.747234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:14.748855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:14.757109 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1250/2511] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:36:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:36:15.338052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262b3db-8622-4a92-b990-505bf9c92961 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.338377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262b3db-8622-4a92-b990-505bf9c92961 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.338775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262b3db-8622-4a92-b990-505bf9c92961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.357161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f262b3db-8622-4a92-b990-505bf9c92961 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:15.358380 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f262b3db-8622-4a92-b990-505bf9c92961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:15.366392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1262/2512] 10.222.0.40 () {58 vars in 967 bytes} [Fri Jul 15 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:15.380935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.390344 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:15.395864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e04935-8dec-48c7-94ae-f35f9b2e4751 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.402247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.402925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.403190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.403475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.412746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.414345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.423134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1251/2513] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:36:15.645196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d570c-ec64-401f-992d-588b72e03781 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.645697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d570c-ec64-401f-992d-588b72e03781 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.646002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d570c-ec64-401f-992d-588b72e03781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.666535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-407d570c-ec64-401f-992d-588b72e03781 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:15.667778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-407d570c-ec64-401f-992d-588b72e03781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:15.678141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1263/2514] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:15.691966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:15.703176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718c76b-fdb1-4506-b845-f844b11263de service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:15.709470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:15.710115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:15.710396 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:15.710739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:15.721057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:15.724553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:15.733349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1252/2515] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:36:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:36:16.118820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:16.119114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:16.119361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:16.138279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:16.139808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:16.148820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1264/2516] 10.222.0.40 () {58 vars in 967 bytes} [Fri Jul 15 15:36:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:16.161764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:16.172462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c8a4-f526-4d81-a740-09caba1524fb service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:16.177787 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911a7b4-77c9-4eb3-92e1-50e717f8fa5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:16.178421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911a7b4-77c9-4eb3-92e1-50e717f8fa5a service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:16.178721 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911a7b4-77c9-4eb3-92e1-50e717f8fa5a service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:16.179003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911a7b4-77c9-4eb3-92e1-50e717f8fa5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:16.187661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911a7b4-77c9-4eb3-92e1-50e717f8fa5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:16.189224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911a7b4-77c9-4eb3-92e1-50e717f8fa5a service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:16.197490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1253/2517] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:36:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:36:18.376057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:18.444498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc720bd2-df62-4bd7-ac07-cae701d2ac3b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:18.449076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:18.449959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.450237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.450508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] PATH_INFO: `/v3/users/d590d4ec416b4e59b1133f3abf86fc7e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.451917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d590d4ec416b4e59b1133f3abf86fc7e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:18.453859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:18.518531 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd590d4ec416b4e59b1133f3abf86fc7e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:18.526344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd590d4ec416b4e59b1133f3abf86fc7e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:18.534968 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd590d4ec416b4e59b1133f3abf86fc7e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:18.535970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd590d4ec416b4e59b1133f3abf86fc7e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:18.536547 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd590d4ec416b4e59b1133f3abf86fc7e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:18.547785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:18.556521 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1265/2518] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:36:18 2022] DELETE /identity/v3/users/d590d4ec416b4e59b1133f3abf86fc7e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:18.561773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.562274 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.562401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.581201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:18.582569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:18.651567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1254/2519] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:36:18.661885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:18.663582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:18.724779 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f60263-dc17-44c1-a577-6a36849c5a2d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:18.729385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dced16-b243-404e-8b54-c0b22f7ed377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:18.730034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dced16-b243-404e-8b54-c0b22f7ed377 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.730315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dced16-b243-404e-8b54-c0b22f7ed377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.730623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dced16-b243-404e-8b54-c0b22f7ed377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.743594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dced16-b243-404e-8b54-c0b22f7ed377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:18.745057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dced16-b243-404e-8b54-c0b22f7ed377 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:18.755664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1266/2520] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:36:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:36:18.959054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:18.961154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:18.975210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bbfe7-c29c-42b0-9bf2-561a507e8645 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:18.981266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:18.982158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:18.982459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:18.982734 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] PATH_INFO: `/v3/projects/af8ddde0deb24e7a9222ec5da477018e` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:18.984049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af8ddde0deb24e7a9222ec5da477018e)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:18.986045 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.037827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8ddde0deb24e7a9222ec5da477018e', 'request_id': 'req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:19.044475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8ddde0deb24e7a9222ec5da477018e', 'request_id': 'req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:19.050123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8ddde0deb24e7a9222ec5da477018e', 'request_id': 'req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:19.064420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] The token cache is being invalidate because project af8ddde0deb24e7a9222ec5da477018e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:36:19.064626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:19.065261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-6f9c983e-3510-40f5-a0f0-a3a3e2c5adf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:36:19.066855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1255/2521] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:36:18 2022] DELETE /identity/v3/projects/af8ddde0deb24e7a9222ec5da477018e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:19.072635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.072927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.073234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.092030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:19.093271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:19.166610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1267/2522] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:19.174816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.175105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.175382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.201309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:19.202545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:19.210881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1256/2523] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:19.215230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.225362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74419938-992c-45d1-a6c7-cc2cf0a5c2d1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.230339 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.231036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.231316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.231583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.232305 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:36:19.232606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.234209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.246630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1268/2524] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:36:19.251032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.260544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50429719-59df-4a3a-85e8-19417b78f632 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.266397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.267209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.267490 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.267761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.268453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.270470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.309146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1257/2525] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:36:19 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:36:19.314664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.326419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1bc3-0e55-4b89-9a5d-2ada5466d05d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.397169 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.397933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.398430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.398742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.399455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.401439 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.434551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1269/2526] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:36:19 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:36:19.440905 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.452509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa787d-46f0-4904-96c9-cef280a07603 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.457857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.458565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.459010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.459443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.460703 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.463747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.469663 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:19.478600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1258/2527] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:36:19.480291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.491031 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c017933-cae2-46fc-ab39-537ce5c5dd31 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.496090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe4a2-aad7-46f9-82d0-9ecafb8a81ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.496753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe4a2-aad7-46f9-82d0-9ecafb8a81ad admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.497030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe4a2-aad7-46f9-82d0-9ecafb8a81ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.497302 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe4a2-aad7-46f9-82d0-9ecafb8a81ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.498026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe4a2-aad7-46f9-82d0-9ecafb8a81ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.499675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe4a2-aad7-46f9-82d0-9ecafb8a81ad admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.510394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1270/2528] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:36:19.514797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.524963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9faa9-c66f-4cc2-ae7b-9b4078c3fe18 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.530012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.530933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.531215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.531483 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] PATH_INFO: `/v3/projects/9bda8b2837a54ec5a75c7dc355571255/users/4453bd7126a849f29cfe3d8bdf4531f1/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.547407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bda8b2837a54ec5a75c7dc355571255, user_id=4453bd7126a849f29cfe3d8bdf4531f1, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.550052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.565806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1259/2529] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:36:19 2022] PUT /identity/v3/projects/9bda8b2837a54ec5a75c7dc355571255/users/4453bd7126a849f29cfe3d8bdf4531f1/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:19.572165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae104e6-21e8-490a-967a-b3559a30379c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.572624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae104e6-21e8-490a-967a-b3559a30379c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.573072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae104e6-21e8-490a-967a-b3559a30379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.583937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:19.596795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7ae104e6-21e8-490a-967a-b3559a30379c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:19.598026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7ae104e6-21e8-490a-967a-b3559a30379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:19.661903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1271/2530] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:19.671579 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:19.673411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:19.744885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb2dea-ffe7-4a5d-b3a0-b753a9a7a793 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:19.749372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:19.749998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:19.750281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:19.750549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:19.761241 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:19.762699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:19.772754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1260/2531] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:36:23.958223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:23.958723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:23.960005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:23.990404 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] MFA Rules not processed for user `4453bd7126a849f29cfe3d8bdf4531f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:23.992279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:24.091853 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1272/2532] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:24.096964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.108278 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31963ba-cbd6-4e9d-b7bc-9e6b116d548e service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.114206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.114925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.115231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.115510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.116343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.118481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.151335 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1261/2533] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:36:24 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:36:24.155721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.167685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bf25f-44f2-42b3-8722-2335e05019d3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.252449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.252449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.252449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.252449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.253136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.255691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.291338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1273/2534] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:36:24 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:36:24.295693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.311153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d4ffa-9d8c-43c7-ace7-cb0af7e1259a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.326248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.327213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.327539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.327855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.328737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.330493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.346690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1262/2535] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:36:24.351896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.369010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99faa65b-c488-4d5e-a350-8bff129aca86 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.376617 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.378251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.379107 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.379558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] PATH_INFO: `/v3/projects/e92a81048fda4c9e91da130fb6135fb8/users/603bc3c0185346fa806d949c98133916/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.403396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92a81048fda4c9e91da130fb6135fb8, user_id=603bc3c0185346fa806d949c98133916, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.407777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.419392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:24.425019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1274/2536] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:36:24 2022] PUT /identity/v3/projects/e92a81048fda4c9e91da130fb6135fb8/users/603bc3c0185346fa806d949c98133916/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:24.430837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.445704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69170ac2-2f54-4da2-b888-3eadc461ad36 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.529638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd1893-36e4-470d-82dd-1dc67ba99bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.530311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1893-36e4-470d-82dd-1dc67ba99bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.530839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1893-36e4-470d-82dd-1dc67ba99bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.531099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd1893-36e4-470d-82dd-1dc67ba99bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.532052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd1893-36e4-470d-82dd-1dc67ba99bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.534410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd1893-36e4-470d-82dd-1dc67ba99bb0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.550600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1263/2537] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:36:24.553778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:24.571950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887483c9-4631-433f-8be8-9d07d7f44921 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:24.577501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:24.578431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:24.578770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:24.579150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] PATH_INFO: `/v3/projects/e92a81048fda4c9e91da130fb6135fb8/users/603bc3c0185346fa806d949c98133916/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:24.581269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e92a81048fda4c9e91da130fb6135fb8, user_id=603bc3c0185346fa806d949c98133916, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:24.584054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:24.596415 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1275/2538] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:36:24 2022] PUT /identity/v3/projects/e92a81048fda4c9e91da130fb6135fb8/users/603bc3c0185346fa806d949c98133916/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:36:29.632810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:29.633459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:29.633459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:29.663502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] MFA Rules not processed for user `603bc3c0185346fa806d949c98133916`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:29.664896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:29.700305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:29.747816 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1264/2539] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:36:29.765840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:29.768889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:29.864561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546179a-a8c7-43ca-84b1-33cf187d788d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:29.875136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:29.875909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:29.876230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:29.876421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:29.958625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:29.960331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:29.971484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1276/2540] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:36:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:36:39.743189 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=15 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:39.966352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:40.233895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:40.235984 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:40.257294 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e2a0f-a547-41f9-895e-263a1fc4cd12 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:40.264328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:40.264999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:40.265283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:40.265561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:40.267826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:40.361842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:40.363480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe39187-1c93-49c9-a107-5276305b1e69 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:40.391869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1265/2541] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:36:43.767402 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:36:43.837793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c08dc-97af-4f09-999a-6f4317129434 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:36:43.841902 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:36:43.842882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:43.843156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:43.843425 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:43.845559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:43.860108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:36:43.861605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:36:43.874226 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1277/2542] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:36:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:36:50.388954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=41 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:53.868519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=26 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:36:58.375726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b03534-0218-49a2-b5eb-1ac713ccf31e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:36:58.375726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b03534-0218-49a2-b5eb-1ac713ccf31e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:36:58.375726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b03534-0218-49a2-b5eb-1ac713ccf31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:36:58.392147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-17b03534-0218-49a2-b5eb-1ac713ccf31e None None] MFA Rules not processed for user `4453bd7126a849f29cfe3d8bdf4531f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:36:58.393394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-17b03534-0218-49a2-b5eb-1ac713ccf31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:36:58.400537 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1266/2543] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:36:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:37:02.434202 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:02.443788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ecd01-16eb-4250-bdfc-baf18ab7aae2 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:02.448377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:02.449015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:02.449291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:02.449559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:02.451517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:02.462761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:02.464946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:02.477148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1278/2544] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:37:07.939638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:07.940180 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:07.940289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:07.995076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:37:07.997257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:08.067843 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1267/2545] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:37:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:37:08.080254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:08.081790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:08.183711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e12fe-43b6-40c0-affb-cd3f2d36d1d0 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:08.189359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:08.189983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:08.190267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:08.190537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:08.199450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:08.201097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:08.209478 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1279/2546] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:37:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:37:14.974780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:14.990041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52deb30-95ee-44c7-b89e-ad877f751920 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:14.997116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:14.997753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:14.998068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:14.998345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:15.000836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:15.104692 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:15.106292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:15.126978 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1268/2547] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:17.914955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:17.926688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22886c08-32d3-403b-a9ed-9e39b6d15392 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:17.934638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:17.934638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:17.934638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:17.934638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:17.953695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:17.955571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:17.967196 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1280/2548] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:37:25.125174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:37:25.530961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:25.532708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:25.600329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da6cb1-a51d-48a6-abf8-ad114656b8f8 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:25.604286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:25.605106 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:25.605484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:25.605863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:25.609266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:25.625164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:25.627211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:25.639724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1269/2549] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:25.722864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:25.724338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:25.730891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=54 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:37:25.736631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5993a1-0d9e-4f89-98de-00a612a684ed service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:25.742655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:25.743498 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:25.743892 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:25.744342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:25.747345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:25.758694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:25.760588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:25.768982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1281/2550] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:25.856477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:25.867580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234ae9f-560f-4b9d-97ba-5de293bca5f2 service glance] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:25.872095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:25.872751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:25.873022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:25.873297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:25.876623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:25.900868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:25.902424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859a71a-0ea0-465b-b201-8fe628dd1394 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:25.915132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1270/2551] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:25.986928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:25.988595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:26.063938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0875cdb3-1860-425d-9709-259c567e9985 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:26.070677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:26.071544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:26.072040 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:26.072385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:26.074580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:26.087561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:26.089393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:26.100583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1282/2552] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:27.754906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:27.755451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:27.755578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:27.774691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:37:27.775910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:27.784108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1271/2553] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:37:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:37:27.797368 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:27.809877 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4df23-c998-44e0-8f38-8f9aa9feb69d service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:27.817009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc9d8a-f248-4404-aef8-50eac4c437e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:27.818506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9d8a-f248-4404-aef8-50eac4c437e0 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:27.818878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9d8a-f248-4404-aef8-50eac4c437e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:27.819252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc9d8a-f248-4404-aef8-50eac4c437e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:27.828830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc9d8a-f248-4404-aef8-50eac4c437e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:27.830458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc9d8a-f248-4404-aef8-50eac4c437e0 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:27.839166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1283/2554] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:37:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:37:37.769416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=35 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:37:37.835747 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=43 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:37:48.102004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:48.122947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8a9f2-c7d6-4b48-b786-ead1684cb768 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:48.128665 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:48.129320 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:48.129652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:48.129951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:48.132780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:48.144810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:48.146401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:48.157624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1272/2555] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:51.466174 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:51.466688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:51.466782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:51.486509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:37:51.487813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:51.496870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1284/2556] 10.222.0.40 () {58 vars in 967 bytes} [Fri Jul 15 15:37:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:37:51.510904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:51.523639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518e21c-4fd7-4922-a8c7-5966abd476b1 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:51.529402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:51.530075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:51.530366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:51.530693 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:51.540217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:51.541818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:51.550596 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1273/2557] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:37:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:37:53.307787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:53.328668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0049ca-ca23-4675-bf78-02c35df51b92 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:53.335601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:53.336280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:53.336655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:53.337208 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:53.339626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:53.360078 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:53.361780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:53.378309 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1285/2558] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:37:58.400709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:37:58.410966 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbada5-8cf8-4e91-90e7-470d70d7ca92 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:37:58.414589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:37:58.415217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:37:58.415527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:37:58.416033 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:37:58.418203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:37:58.486445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:37:58.487928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:37:58.497983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1274/2559] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:02.039280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:02.042643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:02.064077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e37480-7348-4966-9fd1-9b7488b9d8c7 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:02.071170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:02.072517 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.073430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.073913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] PATH_INFO: `/v3/users/08050c1635084492a4b054b172884af8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.075970 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] RBAC: Authorizing `identity:delete_user(user_id=08050c1635084492a4b054b172884af8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:02.079230 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:02.147116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08050c1635084492a4b054b172884af8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.154244 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08050c1635084492a4b054b172884af8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.163873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08050c1635084492a4b054b172884af8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.164512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08050c1635084492a4b054b172884af8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.165228 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6abe0cfe-f646-4176-a46a-b1fe3bd3038c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08050c1635084492a4b054b172884af8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.186047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1286/2560] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:38:02 2022] DELETE /identity/v3/users/08050c1635084492a4b054b172884af8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:02.398211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:02.409583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bc5fc-c596-4398-b43a-450e5f7f8e45 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:02.503768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:02.505277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.505782 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.506240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] PATH_INFO: `/v3/projects/5e51044e804442909f18288b9a6430a5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.508438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e51044e804442909f18288b9a6430a5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:02.512552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:02.579740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e51044e804442909f18288b9a6430a5', 'request_id': 'req-b8cc2e69-a377-4a69-b919-5d909baade88'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.587058 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e51044e804442909f18288b9a6430a5', 'request_id': 'req-b8cc2e69-a377-4a69-b919-5d909baade88'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.594258 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e51044e804442909f18288b9a6430a5', 'request_id': 'req-b8cc2e69-a377-4a69-b919-5d909baade88'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.604452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:02.610114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] The token cache is being invalidate because project 5e51044e804442909f18288b9a6430a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:38:02.610785 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.611625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8cc2e69-a377-4a69-b919-5d909baade88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:02.613799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1275/2561] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:38:02 2022] DELETE /identity/v3/projects/5e51044e804442909f18288b9a6430a5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:38:02.619480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.619845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.620204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.640828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:02.642178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:02.718102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1287/2562] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:02.730918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.731339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.731750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.758737 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:02.760906 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:02.772838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1276/2563] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:02.779080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:02.791727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f1e99-a1b3-4c97-bca1-4577072353c1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:02.796771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:02.797477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.797752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.798018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.798778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:38:02.799080 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:02.800695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:02.813872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1288/2564] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:38:02.818506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:02.829299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f78fa-a56e-4b67-bfcb-f5d92349b692 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:02.834089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:02.834808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.835089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.835362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.836099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:02.838004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:02.866127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1277/2565] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:38:02 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:38:02.870894 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:02.881713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ce495-f4df-4af7-8fe4-a6e986a0017b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:02.923788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:02.950926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:02.951705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:02.952027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:02.952502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:02.953304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:02.955371 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:02.978227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1289/2566] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:38:02 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:38:02.983494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:02.995549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b9d1e-adaf-4497-abbe-bd0bab81c026 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:03.000880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:03.001694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:03.001979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:03.002256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:03.003038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:03.004657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:03.020376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1278/2567] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:03.024733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:03.035957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cffaab-d953-43b8-8005-6244c5f72bfc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:03.041198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09950dc7-68a7-4bae-9179-1eac08158ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:03.042102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09950dc7-68a7-4bae-9179-1eac08158ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:03.042613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09950dc7-68a7-4bae-9179-1eac08158ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:03.043018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09950dc7-68a7-4bae-9179-1eac08158ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:03.043866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09950dc7-68a7-4bae-9179-1eac08158ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:03.045623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09950dc7-68a7-4bae-9179-1eac08158ef2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:03.057603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1290/2568] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:03.062500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:03.073085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa113051-7ef2-4bd7-9528-678bc423346f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:03.078023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:03.078949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:03.079231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:03.079504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] PATH_INFO: `/v3/projects/db2959b3dfd24412a793d60ba82c6deb/users/8e36c74bcd0c4132aa2f159a0152cd78/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:03.097306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db2959b3dfd24412a793d60ba82c6deb, user_id=8e36c74bcd0c4132aa2f159a0152cd78, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:03.100405 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:03.116588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1279/2569] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:38:03 2022] PUT /identity/v3/projects/db2959b3dfd24412a793d60ba82c6deb/users/8e36c74bcd0c4132aa2f159a0152cd78/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:03.128074 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:03.128414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:03.128777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:03.158407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] MFA Rules not processed for user `8e36c74bcd0c4132aa2f159a0152cd78`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:03.159647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:03.228341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1291/2570] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:03.240979 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:03.242850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:03.312224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc093a59-9038-493c-b0ce-ab495a83e8b4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:03.316388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429aebfe-df6a-4601-a6bf-2db5361334fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:03.317097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429aebfe-df6a-4601-a6bf-2db5361334fa service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:03.317420 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429aebfe-df6a-4601-a6bf-2db5361334fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:03.317731 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429aebfe-df6a-4601-a6bf-2db5361334fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:03.327981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429aebfe-df6a-4601-a6bf-2db5361334fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:03.329434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429aebfe-df6a-4601-a6bf-2db5361334fa service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:03.341000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1280/2571] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:13.089245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:13.188501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439198fd-bbfe-4769-83de-f0283b981813 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:13.195568 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:13.196456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:13.196697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:13.197111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] PATH_INFO: `/v3/users/8e36c74bcd0c4132aa2f159a0152cd78` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:13.198238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e36c74bcd0c4132aa2f159a0152cd78)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:13.200240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:13.256198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e36c74bcd0c4132aa2f159a0152cd78'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:13.258280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:13.266164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e36c74bcd0c4132aa2f159a0152cd78'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:13.276845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e36c74bcd0c4132aa2f159a0152cd78'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:13.277713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e36c74bcd0c4132aa2f159a0152cd78'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:13.278544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e36c74bcd0c4132aa2f159a0152cd78'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:13.297495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1292/2572] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:38:13 2022] DELETE /identity/v3/users/8e36c74bcd0c4132aa2f159a0152cd78 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:13.302199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:13.302750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:13.302869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:13.321439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:13.323100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:13.387269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1281/2573] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:13.397785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:13.399540 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:13.468112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f09f0f-8066-4542-8317-9f865b3d1b3b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:13.474148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12bc429-5c01-40d1-ae0f-8678a8eb8b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:13.474862 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bc429-5c01-40d1-ae0f-8678a8eb8b9e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:13.475141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bc429-5c01-40d1-ae0f-8678a8eb8b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:13.475432 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12bc429-5c01-40d1-ae0f-8678a8eb8b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:13.486512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bc429-5c01-40d1-ae0f-8678a8eb8b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:13.488000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12bc429-5c01-40d1-ae0f-8678a8eb8b9e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:13.498266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1293/2574] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:14.025133 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.026844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:14.046904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878b74c-c189-457a-bb6a-4e739eeb0c12 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.052426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.053326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.053598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.053920 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] PATH_INFO: `/v3/projects/db2959b3dfd24412a793d60ba82c6deb` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.058562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=db2959b3dfd24412a793d60ba82c6deb)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.058756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.104553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:14.121997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db2959b3dfd24412a793d60ba82c6deb', 'request_id': 'req-461af642-b112-41c7-a698-e229c60582ab'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:14.128968 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db2959b3dfd24412a793d60ba82c6deb', 'request_id': 'req-461af642-b112-41c7-a698-e229c60582ab'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:14.134756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db2959b3dfd24412a793d60ba82c6deb', 'request_id': 'req-461af642-b112-41c7-a698-e229c60582ab'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:14.152370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] The token cache is being invalidate because project db2959b3dfd24412a793d60ba82c6deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:38:14.152700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:14.153690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-461af642-b112-41c7-a698-e229c60582ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:14.155755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1282/2575] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:38:14 2022] DELETE /identity/v3/projects/db2959b3dfd24412a793d60ba82c6deb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:14.160805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.161303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.161400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.179958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:14.181189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:14.259788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1294/2576] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:14.273290 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.273657 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.273899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.297674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:14.298950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:14.307615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1283/2577] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:14.312418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.322560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b474271-fe71-4361-9a0a-8f0b5110f87c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.327742 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.328406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.328683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.328954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.329679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:38:14.329980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.331626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.344198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1295/2578] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:38:14.348941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.359214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9fe5d3-5371-42b3-9b5c-12eaa8a4d465 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.365784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.366492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.366877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.367155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.368223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.370151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.397909 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1284/2579] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:38:14 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:38:14.402340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.414764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cac0c4-a2cc-4608-982e-e41597c8725d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.480337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:14.488082 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.488791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.489364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.489648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.490401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.493191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.515833 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1296/2580] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:38:14 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:38:14.519074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.530072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c43a32f-ac9c-4452-bad6-2f55bc77f91d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.535129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.535812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.536094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.536369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.537105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.538784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.549936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1285/2581] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:14.554408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.566981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef60c6-f43a-4d22-bd19-53996fb5e3bc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.572693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380fb1d-4c48-451f-90c6-293131fa59c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.573376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380fb1d-4c48-451f-90c6-293131fa59c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.573659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380fb1d-4c48-451f-90c6-293131fa59c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.573933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380fb1d-4c48-451f-90c6-293131fa59c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.574708 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380fb1d-4c48-451f-90c6-293131fa59c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.576761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380fb1d-4c48-451f-90c6-293131fa59c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.587666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1297/2582] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:14.592253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.602283 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a900f-7070-4d8c-96d3-fb48c1ca0b2c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.607092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.608020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.608425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.608672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] PATH_INFO: `/v3/projects/c1daf9239aeb40c4a61a14de527dc954/users/ca12aa795dd94e5aa83365cb096ca3ac/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.628141 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1daf9239aeb40c4a61a14de527dc954, user_id=ca12aa795dd94e5aa83365cb096ca3ac, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.630974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.646826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1286/2583] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:38:14 2022] PUT /identity/v3/projects/c1daf9239aeb40c4a61a14de527dc954/users/ca12aa795dd94e5aa83365cb096ca3ac/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:14.658464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.658668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.658964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.688105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] MFA Rules not processed for user `ca12aa795dd94e5aa83365cb096ca3ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:14.689334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:14.762231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1298/2584] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:14.775339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:14.777381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:14.857714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bbc10-bda4-4ee3-849b-8962f7317e5e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:14.863837 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5726349-362a-4165-914a-f5b506ce99bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:14.864498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726349-362a-4165-914a-f5b506ce99bb service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:14.864763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726349-362a-4165-914a-f5b506ce99bb service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:14.865047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5726349-362a-4165-914a-f5b506ce99bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:14.874282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5726349-362a-4165-914a-f5b506ce99bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:14.875934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5726349-362a-4165-914a-f5b506ce99bb service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:14.885942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1287/2585] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:38:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:15.112590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:15.187176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc76a07-039f-4e71-912f-2a1fd84e9e95 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:15.192471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:15.193395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:15.193718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:15.193980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] PATH_INFO: `/v3/users/ca12aa795dd94e5aa83365cb096ca3ac` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:15.195287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca12aa795dd94e5aa83365cb096ca3ac)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:15.197308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:15.254502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca12aa795dd94e5aa83365cb096ca3ac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.262068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca12aa795dd94e5aa83365cb096ca3ac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.271694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca12aa795dd94e5aa83365cb096ca3ac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.272396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca12aa795dd94e5aa83365cb096ca3ac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.273127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca12aa795dd94e5aa83365cb096ca3ac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.278292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:15.295515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1299/2586] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:38:15 2022] DELETE /identity/v3/users/ca12aa795dd94e5aa83365cb096ca3ac => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:15.300479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:15.300777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:15.301039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:15.320494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:15.321756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:15.364857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:15.390934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1288/2587] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:15.400907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:15.402442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:15.469845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae430fa-3ef8-4967-b0a5-507996597308 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:15.474272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5fe91-f91a-47a6-9e2e-2b72bbc4cfeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:15.475012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5fe91-f91a-47a6-9e2e-2b72bbc4cfeb service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:15.475278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5fe91-f91a-47a6-9e2e-2b72bbc4cfeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:15.475538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5fe91-f91a-47a6-9e2e-2b72bbc4cfeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:15.493480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5fe91-f91a-47a6-9e2e-2b72bbc4cfeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:15.494995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5fe91-f91a-47a6-9e2e-2b72bbc4cfeb service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:15.506949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1300/2588] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:38:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:15.835644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:15.837216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:15.851028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2b846-8b4a-4836-b50d-116b4770efac None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:15.857083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:15.858077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:15.858361 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:15.858708 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] PATH_INFO: `/v3/projects/c1daf9239aeb40c4a61a14de527dc954` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:15.860005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1daf9239aeb40c4a61a14de527dc954)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:15.861967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:15.913039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1daf9239aeb40c4a61a14de527dc954', 'request_id': 'req-860fffb3-7129-45ae-9275-797d6c02601c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.925326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1daf9239aeb40c4a61a14de527dc954', 'request_id': 'req-860fffb3-7129-45ae-9275-797d6c02601c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.931564 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1daf9239aeb40c4a61a14de527dc954', 'request_id': 'req-860fffb3-7129-45ae-9275-797d6c02601c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.954866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] The token cache is being invalidate because project c1daf9239aeb40c4a61a14de527dc954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:38:15.954866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.954866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-860fffb3-7129-45ae-9275-797d6c02601c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:38:15.954866 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1289/2589] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:38:15 2022] DELETE /identity/v3/projects/c1daf9239aeb40c4a61a14de527dc954 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:15.958239 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:15.958558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:15.958844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:15.977356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:15.978626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:16.044111 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1301/2590] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:16.053087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.053380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.053650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.075025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:16.076280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:16.085286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1290/2591] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:16.093718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.106168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e8cfd-1824-4c48-9820-2d3e0727cd9e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.113567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.114555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.115201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.115623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.116787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:38:16.117259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.120381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.135155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1302/2592] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:38:16.142696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.156608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484960a-0c2f-41a8-8bdc-70e8a339d88d None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.162952 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.163933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.164356 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.164768 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.165856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.168934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.205366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1291/2593] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:38:16 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:38:16.209784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.223194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb2d1c-2851-4c63-b1ab-003955d82400 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.286834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.287513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.287797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.288075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.288773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.290773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.313316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1303/2594] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:38:16 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:38:16.317517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.329893 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e583c-b603-4cbc-b0e2-13fe1b03b2da admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.336028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.336700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.336976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.337221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.337951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.339584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.352032 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1292/2595] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:38:16.357132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.363826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:16.368127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dce921-fdec-423e-8c9e-14b7d9970756 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.374613 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452fee3-7cde-4234-b437-9eb38896206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.375687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452fee3-7cde-4234-b437-9eb38896206b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.376200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452fee3-7cde-4234-b437-9eb38896206b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.376554 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452fee3-7cde-4234-b437-9eb38896206b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.378203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452fee3-7cde-4234-b437-9eb38896206b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.380460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452fee3-7cde-4234-b437-9eb38896206b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.394799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1304/2596] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:16.399583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.411487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b54f0-9a17-49c0-be3e-a296f1017df5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.417158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.418120 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.418406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.418727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] PATH_INFO: `/v3/projects/3140aca0850b4bfcb1470a449f9065cd/users/85d22d82ea9c456ea37bfbbb7443725c/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.437963 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3140aca0850b4bfcb1470a449f9065cd, user_id=85d22d82ea9c456ea37bfbbb7443725c, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.441135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.457552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1293/2597] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:38:16 2022] PUT /identity/v3/projects/3140aca0850b4bfcb1470a449f9065cd/users/85d22d82ea9c456ea37bfbbb7443725c/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:16.478069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.478509 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.479017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.518168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] MFA Rules not processed for user `85d22d82ea9c456ea37bfbbb7443725c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:16.519467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:16.593718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1305/2598] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:16.599007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.666455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a6f4-71c9-43e7-9138-a2a0d4c7fffd admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.672215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.673000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.673395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.673556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.674261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.676419 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.681197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:16.707960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1294/2599] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:38:16 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:38:16.712189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.724280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656710a0-b223-4457-9ad8-d7997d75db66 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.789618 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.790718 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.791117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.791428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.792468 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.795391 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.822484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1306/2600] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:38:16 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:38:16.827501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.838702 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db308f-7fe4-4bb3-92f7-bce11405d061 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.844754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.845358 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.845634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.845898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.846642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.848238 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.860600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1295/2601] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:16.865083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.876772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ea903-e88b-4b8a-81f0-b1e6846b7e80 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:16.883875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:16.884795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:16.885077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:16.885363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] PATH_INFO: `/v3/projects/574cfbbebb1046edafce5940fc7b908f/users/3a97646ec4374d05826b5dfe05bcee8a/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:16.902589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574cfbbebb1046edafce5940fc7b908f, user_id=3a97646ec4374d05826b5dfe05bcee8a, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:16.905621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:16.919991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1307/2602] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:38:16 2022] PUT /identity/v3/projects/574cfbbebb1046edafce5940fc7b908f/users/3a97646ec4374d05826b5dfe05bcee8a/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:16.924272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:16.934758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea91b6-ef18-4c79-b5f4-838e9049daea admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:17.002748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae63fba-553a-4715-8a96-f21d823bb59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:17.003424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae63fba-553a-4715-8a96-f21d823bb59f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:17.003711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae63fba-553a-4715-8a96-f21d823bb59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:17.003993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae63fba-553a-4715-8a96-f21d823bb59f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:17.004727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae63fba-553a-4715-8a96-f21d823bb59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:17.006333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae63fba-553a-4715-8a96-f21d823bb59f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:17.017225 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1296/2603] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:38:17.022995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:17.035812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc93bf9-370d-4522-b03e-b89554b32944 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:17.041937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:17.042868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:17.043150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:17.043417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] PATH_INFO: `/v3/projects/574cfbbebb1046edafce5940fc7b908f/users/3a97646ec4374d05826b5dfe05bcee8a/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:17.045909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574cfbbebb1046edafce5940fc7b908f, user_id=3a97646ec4374d05826b5dfe05bcee8a, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:17.048739 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:17.060630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1308/2604] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:38:17 2022] PUT /identity/v3/projects/574cfbbebb1046edafce5940fc7b908f/users/3a97646ec4374d05826b5dfe05bcee8a/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:38:17.071551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:17.071836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:17.072108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:17.102975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] MFA Rules not processed for user `3a97646ec4374d05826b5dfe05bcee8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:38:17.104147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:17.183880 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1297/2605] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:38:17.197917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:17.200019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:38:17.244450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:17.267755 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b9c3d-b861-49b9-b13e-89259a2f5450 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:17.272406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:17.273274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:17.273555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:17.273830 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:17.284731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:17.286236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:17.296389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1309/2606] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:17.375719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:38:17.386774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e449a977-8f8c-4f44-9890-e665c1ed0a24 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:38:17.393515 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298cc899-7fbe-4859-9652-e29915451921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:38:17.394424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cc899-7fbe-4859-9652-e29915451921 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:38:17.394761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cc899-7fbe-4859-9652-e29915451921 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:38:17.395030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cc899-7fbe-4859-9652-e29915451921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:38:17.456331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:17.458876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cc899-7fbe-4859-9652-e29915451921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:38:17.460765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298cc899-7fbe-4859-9652-e29915451921 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:38:17.472390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1298/2607] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:38:27.291706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:38:27.469414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:03.716130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:03.794629 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b731c0e-b0e2-49ea-9e61-49e08ca5fead service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:03.799913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:03.800933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:03.801249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:03.801575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] PATH_INFO: `/v3/users/85d22d82ea9c456ea37bfbbb7443725c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:03.802956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d22d82ea9c456ea37bfbbb7443725c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:03.805284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:03.861651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d22d82ea9c456ea37bfbbb7443725c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:03.872398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d22d82ea9c456ea37bfbbb7443725c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:03.882544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d22d82ea9c456ea37bfbbb7443725c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:03.883288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d22d82ea9c456ea37bfbbb7443725c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:03.883914 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d22d82ea9c456ea37bfbbb7443725c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:03.906292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1310/2608] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:03 2022] DELETE /identity/v3/users/85d22d82ea9c456ea37bfbbb7443725c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:03.911041 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:03.911624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:03.911624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:03.931496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:03.933295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:03.997544 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1299/2609] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:04.007666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:04.009266 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:04.072191 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a5e32-63ba-4db3-b8c5-7799f02706a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:04.076433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:04.077128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:04.077410 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:04.077736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:04.088000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:04.089691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:04.101289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1311/2610] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:04.425295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:04.427844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:04.444052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9fd329-9a4f-4e2e-b095-687bc0acfacd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:04.450179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:04.451132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:04.451412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:04.451688 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] PATH_INFO: `/v3/projects/3140aca0850b4bfcb1470a449f9065cd` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:04.452941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3140aca0850b4bfcb1470a449f9065cd)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:04.454991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:04.518568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3140aca0850b4bfcb1470a449f9065cd', 'request_id': 'req-9250f678-df81-4f7c-ab0e-02c139d64b78'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.526251 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3140aca0850b4bfcb1470a449f9065cd', 'request_id': 'req-9250f678-df81-4f7c-ab0e-02c139d64b78'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.533025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3140aca0850b4bfcb1470a449f9065cd', 'request_id': 'req-9250f678-df81-4f7c-ab0e-02c139d64b78'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.551221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] The token cache is being invalidate because project 3140aca0850b4bfcb1470a449f9065cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:04.551567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.552322 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.554057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1300/2611] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:04 2022] DELETE /identity/v3/projects/3140aca0850b4bfcb1470a449f9065cd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:04.559206 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:04.571533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765ccf9-9e7c-43f9-b3f3-bd44f1c99293 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:04.592283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:04.637834 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:04.638750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:04.639036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:04.639296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] PATH_INFO: `/v3/users/3a97646ec4374d05826b5dfe05bcee8a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:04.640473 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a97646ec4374d05826b5dfe05bcee8a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:04.642408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:04.700190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a97646ec4374d05826b5dfe05bcee8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.709013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a97646ec4374d05826b5dfe05bcee8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.718932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a97646ec4374d05826b5dfe05bcee8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.719963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a97646ec4374d05826b5dfe05bcee8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.721162 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e25ccf24-d6fc-4bca-9158-a0e0466e042b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a97646ec4374d05826b5dfe05bcee8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:04.742227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1312/2612] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:04 2022] DELETE /identity/v3/users/3a97646ec4374d05826b5dfe05bcee8a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:05.043856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.045765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:05.062210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250f678-df81-4f7c-ab0e-02c139d64b78 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.122917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.123695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.123975 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.124244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] PATH_INFO: `/v3/projects/574cfbbebb1046edafce5940fc7b908f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.125467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=574cfbbebb1046edafce5940fc7b908f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.127416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.160043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:05.177756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574cfbbebb1046edafce5940fc7b908f', 'request_id': 'req-635f4d88-d25b-43fd-9979-3d8e0e68be8c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:05.184481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574cfbbebb1046edafce5940fc7b908f', 'request_id': 'req-635f4d88-d25b-43fd-9979-3d8e0e68be8c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:05.190597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574cfbbebb1046edafce5940fc7b908f', 'request_id': 'req-635f4d88-d25b-43fd-9979-3d8e0e68be8c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:05.204388 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] The token cache is being invalidate because project 574cfbbebb1046edafce5940fc7b908f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:05.204691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:05.205620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-635f4d88-d25b-43fd-9979-3d8e0e68be8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:05.207345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1301/2613] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:05 2022] DELETE /identity/v3/projects/574cfbbebb1046edafce5940fc7b908f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:39:05.212654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.213064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.214048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.237839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:05.239182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:05.305290 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1313/2614] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:05.313239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.313521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.313807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.337842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:05.338831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:05.349063 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1302/2615] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:05.354438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.366964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1fe1a3-941d-4083-899f-2778927a196e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.373635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.374413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.374832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.375170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.376008 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:39:05.376418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.378398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.393653 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1314/2616] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:39:05.398499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.408790 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1f50a-b579-4c0e-888d-da4a31a6fd59 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.413833 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.414496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.414840 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.415150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.415897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.418505 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.447855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1303/2617] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:05.452980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.463500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d9be2-61ec-4665-8d3d-0a99008ed9b7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.490548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:05.526541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.527349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.527701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.528053 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.528865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.531124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.554890 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1315/2618] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:05.559287 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.570209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882722a9-dd87-418f-aa64-70c3e11dd19f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.575444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.576116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.576399 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.576667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.577423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.579093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.592772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1304/2619] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:05.596906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.608596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6e57e-0371-46c6-a8e2-0506e4c80898 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.614350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4293db-50c2-4807-881d-d20b3a58658f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.615085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4293db-50c2-4807-881d-d20b3a58658f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.615431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4293db-50c2-4807-881d-d20b3a58658f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.615693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4293db-50c2-4807-881d-d20b3a58658f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.616443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4293db-50c2-4807-881d-d20b3a58658f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.618084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4293db-50c2-4807-881d-d20b3a58658f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.631450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1316/2620] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:05.636269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.647353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32194906-6228-46e6-8b47-c0b5417a58e3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.653870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.655095 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.655378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.655712 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] PATH_INFO: `/v3/projects/2cce2ce050e742719b182f54a48c2ed1/users/5f951ea583c64177b91b9ab45f531645/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.673694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cce2ce050e742719b182f54a48c2ed1, user_id=5f951ea583c64177b91b9ab45f531645, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.676876 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.693492 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1305/2621] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:05 2022] PUT /identity/v3/projects/2cce2ce050e742719b182f54a48c2ed1/users/5f951ea583c64177b91b9ab45f531645/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:05.706284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.706967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.707942 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.738275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] MFA Rules not processed for user `5f951ea583c64177b91b9ab45f531645`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:05.739603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:05.825768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1317/2622] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:05.831065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.903525 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61ea12-b203-44f3-be22-d6ac713b4857 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:05.908211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:05.908874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:05.909158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:05.909432 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:05.910123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:05.912070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:05.945494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1306/2623] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:05.946950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:05.960992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e65674-1493-441a-8944-f2385a8a04e1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.023231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.023907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.024187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.024456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.025148 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.027418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.052566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1318/2624] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:05 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:06.056858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:06.068246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2cf24-ff1e-457e-be14-80ea25a660e9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.073557 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.074248 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.074563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.074832 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.075560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.077155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.087912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1307/2625] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:06.092370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:06.102838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c398f7-e241-41f1-ac03-70e4db30849d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.107976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.108854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.109128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.109412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] PATH_INFO: `/v3/projects/517431a1a94a412eaa58104eb61b7cb3/users/a4dced09b3a24bd1b6da4530a5d8a14e/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.125501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517431a1a94a412eaa58104eb61b7cb3, user_id=a4dced09b3a24bd1b6da4530a5d8a14e, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.128998 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.143224 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1319/2626] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:06 2022] PUT /identity/v3/projects/517431a1a94a412eaa58104eb61b7cb3/users/a4dced09b3a24bd1b6da4530a5d8a14e/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:06.147893 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:06.159581 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a760-cff7-428e-8746-7fdf176c90f6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.181329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:06.224353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc7764-d1f6-4533-8a27-479b44381bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.225035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc7764-d1f6-4533-8a27-479b44381bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.225325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc7764-d1f6-4533-8a27-479b44381bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.225601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc7764-d1f6-4533-8a27-479b44381bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.226407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc7764-d1f6-4533-8a27-479b44381bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.228261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc7764-d1f6-4533-8a27-479b44381bfa admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.239709 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1308/2627] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:06.244098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:06.249651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:06.257137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177719c6-66ea-4c2b-a8da-546157aae510 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.263398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.264232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.264636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.264924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] PATH_INFO: `/v3/projects/517431a1a94a412eaa58104eb61b7cb3/users/a4dced09b3a24bd1b6da4530a5d8a14e/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.267010 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517431a1a94a412eaa58104eb61b7cb3, user_id=a4dced09b3a24bd1b6da4530a5d8a14e, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.269707 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.281802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1320/2628] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:06 2022] PUT /identity/v3/projects/517431a1a94a412eaa58104eb61b7cb3/users/a4dced09b3a24bd1b6da4530a5d8a14e/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:06.293536 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.293826 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.294108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.322859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] MFA Rules not processed for user `a4dced09b3a24bd1b6da4530a5d8a14e`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:06.324512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:06.407472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1309/2629] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:06.419606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:06.421609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:06.481561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ade0e-763e-4fbb-b108-2f653f377e68 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.485296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.485943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.486223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.486492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.496535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.497989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.507772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1321/2630] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:06.583083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:06.595767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600111ad-34eb-4e14-b0ec-e9e8bdd0f070 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:06.600982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b86200-a7de-45bd-bc37-4aed17b8ccad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:06.601838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b86200-a7de-45bd-bc37-4aed17b8ccad service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:06.602231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b86200-a7de-45bd-bc37-4aed17b8ccad service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:06.602612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b86200-a7de-45bd-bc37-4aed17b8ccad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:06.666137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b86200-a7de-45bd-bc37-4aed17b8ccad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:06.667634 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b86200-a7de-45bd-bc37-4aed17b8ccad service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:06.678372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1310/2631] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:16.503476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:16.673879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=80 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:16.737296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:16.876435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2340a-4fbc-4856-b031-51efb29a109a service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:16.886218 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:16.887237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:16.887669 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:16.887954 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] PATH_INFO: `/v3/users/5f951ea583c64177b91b9ab45f531645` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:16.890151 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f951ea583c64177b91b9ab45f531645)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:16.892447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:16.973277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f951ea583c64177b91b9ab45f531645'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:16.990737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f951ea583c64177b91b9ab45f531645'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.002359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f951ea583c64177b91b9ab45f531645'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.003334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f951ea583c64177b91b9ab45f531645'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.004086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f951ea583c64177b91b9ab45f531645'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.030733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1322/2632] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:16 2022] DELETE /identity/v3/users/5f951ea583c64177b91b9ab45f531645 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:17.035666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:17.036182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:17.036459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:17.058077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:17.059767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:17.170227 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1311/2633] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:39:17.188959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:17.190693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:17.326462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa60a7c-c0e3-4826-b034-1660b3016211 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:17.335957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:17.336697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:17.337012 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:17.337296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:17.353520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:17.356726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:17.376281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1323/2634] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:17.690486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:17.692760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:17.710375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30963089-5c1b-4ca0-955f-91d6c0865b09 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:17.720366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:17.723424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:17.723956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:17.724457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] PATH_INFO: `/v3/projects/2cce2ce050e742719b182f54a48c2ed1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:17.727529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cce2ce050e742719b182f54a48c2ed1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:17.731620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:17.789998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cce2ce050e742719b182f54a48c2ed1', 'request_id': 'req-71bf58c1-cc39-4111-987c-700f750a6f6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.798670 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cce2ce050e742719b182f54a48c2ed1', 'request_id': 'req-71bf58c1-cc39-4111-987c-700f750a6f6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.805615 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cce2ce050e742719b182f54a48c2ed1', 'request_id': 'req-71bf58c1-cc39-4111-987c-700f750a6f6b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.823446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] The token cache is being invalidate because project 2cce2ce050e742719b182f54a48c2ed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:17.823769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.824748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.828165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1312/2635] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:17 2022] DELETE /identity/v3/projects/2cce2ce050e742719b182f54a48c2ed1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:17.835133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:17.845899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f6660-169a-4d90-bba5-7b3a0cf5a5b5 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:17.868443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:17.916991 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:17.918123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:17.918406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:17.918832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] PATH_INFO: `/v3/users/a4dced09b3a24bd1b6da4530a5d8a14e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:17.920978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4dced09b3a24bd1b6da4530a5d8a14e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:17.924044 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:17.989651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4dced09b3a24bd1b6da4530a5d8a14e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:17.997054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4dced09b3a24bd1b6da4530a5d8a14e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.004938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4dced09b3a24bd1b6da4530a5d8a14e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.005773 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4dced09b3a24bd1b6da4530a5d8a14e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.006394 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-52840884-477f-4337-a9c8-d04d180fd42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4dced09b3a24bd1b6da4530a5d8a14e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.027873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1324/2636] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:17 2022] DELETE /identity/v3/users/a4dced09b3a24bd1b6da4530a5d8a14e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:18.352286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:18.354393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:18.378170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf58c1-cc39-4111-987c-700f750a6f6b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:18.458068 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:18.460458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.460458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.460458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] PATH_INFO: `/v3/projects/517431a1a94a412eaa58104eb61b7cb3` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.461637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=517431a1a94a412eaa58104eb61b7cb3)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:18.463994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:18.502104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:18.525202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517431a1a94a412eaa58104eb61b7cb3', 'request_id': 'req-2b23a513-0985-475d-99da-bf8a621c0f6c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.532365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517431a1a94a412eaa58104eb61b7cb3', 'request_id': 'req-2b23a513-0985-475d-99da-bf8a621c0f6c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.538874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517431a1a94a412eaa58104eb61b7cb3', 'request_id': 'req-2b23a513-0985-475d-99da-bf8a621c0f6c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.554779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] The token cache is being invalidate because project 517431a1a94a412eaa58104eb61b7cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:18.555165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.556076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-2b23a513-0985-475d-99da-bf8a621c0f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:18.558306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1313/2637] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:18 2022] DELETE /identity/v3/projects/517431a1a94a412eaa58104eb61b7cb3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:18.565528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.565796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.566103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.587396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:18.588592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:18.681126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1325/2638] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:18.689668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.689957 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.690214 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.709807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:18.711050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:18.719486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1314/2639] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:18.724163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:18.734185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222db818-872d-45d1-8ea4-0a67e4809912 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:18.743185 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:18.744141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.744504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.744875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.745841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:39:18.746220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:18.749223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:18.766823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1326/2640] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:39:18.771327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:18.781683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ed89-8b66-4381-8881-f8aaf1ebdf7a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:18.787418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:18.788415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.788823 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.789255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.790054 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:18.792182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:18.823233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1315/2641] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:18 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:18.828632 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:18.840500 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21e2be-2420-4f32-849f-fe756aade89c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:18.868549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:18.905958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:18.906679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.906966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.907245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.907949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:18.909904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:18.936926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1327/2642] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:18 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:18.939625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:18.956527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd9b74-db0e-44ee-a7ed-a317e74d6bf2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:18.962211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:18.963072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:18.963381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:18.963659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:18.964393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:18.966028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:18.978090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1316/2643] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:18.982563 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:18.997301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a27a4f0-996b-4cbf-ad4e-1f9a86d43bbf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:19.003537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5a8d1-97c7-4050-86e6-f8ee10deda02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:19.004319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5a8d1-97c7-4050-86e6-f8ee10deda02 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.004674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5a8d1-97c7-4050-86e6-f8ee10deda02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.004989 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5a8d1-97c7-4050-86e6-f8ee10deda02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.005939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5a8d1-97c7-4050-86e6-f8ee10deda02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:19.008076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5a8d1-97c7-4050-86e6-f8ee10deda02 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:19.025263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1328/2644] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:19.026462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:19.039108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d2062-3a83-4683-8dc8-5d1409d9bd74 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:19.045881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:19.046901 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.047243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.047553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] PATH_INFO: `/v3/projects/6c31477dc08b489fa7c7e79667e7fad5/users/f6c9d626f73545f8ad3b22c2e8deef58/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.066477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c31477dc08b489fa7c7e79667e7fad5, user_id=f6c9d626f73545f8ad3b22c2e8deef58, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:19.069254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:19.086757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1317/2645] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:19 2022] PUT /identity/v3/projects/6c31477dc08b489fa7c7e79667e7fad5/users/f6c9d626f73545f8ad3b22c2e8deef58/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:39:19.099021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.099392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.099737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.133054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] MFA Rules not processed for user `f6c9d626f73545f8ad3b22c2e8deef58`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:19.134281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:19.223514 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1329/2646] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:39:19.243679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:19.245855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:19.313223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f699-c763-480d-bf07-39c6a1bd310c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:19.316971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d419d64-3ab0-4072-8864-4c3a8082ba35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:19.317633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419d64-3ab0-4072-8864-4c3a8082ba35 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.317923 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419d64-3ab0-4072-8864-4c3a8082ba35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.318190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d419d64-3ab0-4072-8864-4c3a8082ba35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.328020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d419d64-3ab0-4072-8864-4c3a8082ba35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:19.329459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d419d64-3ab0-4072-8864-4c3a8082ba35 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:19.340098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1318/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:19.362631 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:19.441093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afdf3e-1039-4998-98e2-e3153d7e4c8a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:19.446655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:19.447666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.447988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.448284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] PATH_INFO: `/v3/users/f6c9d626f73545f8ad3b22c2e8deef58` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.449726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c9d626f73545f8ad3b22c2e8deef58)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:19.452435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:19.500267 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:19.508737 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c9d626f73545f8ad3b22c2e8deef58'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:19.518624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c9d626f73545f8ad3b22c2e8deef58'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:19.528396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c9d626f73545f8ad3b22c2e8deef58'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:19.529067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c9d626f73545f8ad3b22c2e8deef58'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:19.529775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c9d626f73545f8ad3b22c2e8deef58'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:19.556221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1330/2648] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:19 2022] DELETE /identity/v3/users/f6c9d626f73545f8ad3b22c2e8deef58 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:19.560911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.561198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.561472 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.580705 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:19.581855 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:19.631155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:19.647029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1319/2649] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:19.657482 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:19.659058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:19.724466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf78162-3ec1-4d84-b859-47ef85681dba admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:19.729465 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391900b-2986-4151-b362-a9ff2de9d5ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:19.730188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391900b-2986-4151-b362-a9ff2de9d5ae service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:19.730467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391900b-2986-4151-b362-a9ff2de9d5ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:19.730766 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391900b-2986-4151-b362-a9ff2de9d5ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:19.742816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391900b-2986-4151-b362-a9ff2de9d5ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:19.744313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391900b-2986-4151-b362-a9ff2de9d5ae service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:19.754798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1331/2650] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:20.057854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.060346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:20.080131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c9d56-24dd-4726-8fc1-3e5f494b04b1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.089321 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.091135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.091660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.092267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] PATH_INFO: `/v3/projects/6c31477dc08b489fa7c7e79667e7fad5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.094341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c31477dc08b489fa7c7e79667e7fad5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.098329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.164714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c31477dc08b489fa7c7e79667e7fad5', 'request_id': 'req-071038a9-d649-476f-8547-1966c9111e55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:20.171983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c31477dc08b489fa7c7e79667e7fad5', 'request_id': 'req-071038a9-d649-476f-8547-1966c9111e55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:20.177804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c31477dc08b489fa7c7e79667e7fad5', 'request_id': 'req-071038a9-d649-476f-8547-1966c9111e55'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:20.192363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] The token cache is being invalidate because project 6c31477dc08b489fa7c7e79667e7fad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:20.192636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:20.193346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-071038a9-d649-476f-8547-1966c9111e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:20.195108 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1320/2651] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:20 2022] DELETE /identity/v3/projects/6c31477dc08b489fa7c7e79667e7fad5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:20.200666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.201584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.201584 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.223135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:20.224768 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:20.323549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1332/2652] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:20.333609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.333999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.334366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.365831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:20.367720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:20.380132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1321/2653] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:20.386124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.413043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b95436-b3a8-4412-9856-0c6350bcb6ea None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.421238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.422246 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.422975 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.423602 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.425030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:39:20.425598 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.431390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.447657 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1333/2654] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:39:20.453858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.468584 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e08b5b-263d-4453-abaa-d8e1d598e4a6 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.475569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.476476 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.477297 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.477872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.478845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.480801 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.516317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1322/2655] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:20 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:20.521018 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.532407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c1d26-1fb1-40f8-a491-3ffb7a55abfb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.588820 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:20.606428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.607420 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.607980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.608334 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.609277 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.611790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.639713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1334/2656] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:20 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:39:20.644616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.655936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c1673-e570-42ec-94eb-4795460ada1f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.660918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.661598 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.661877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.662151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.662921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.664517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.677244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1323/2657] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:20.682395 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.696663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca372dcb-ae6e-4529-8481-0ae2aee21032 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.703273 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d76344-1fd3-4bae-b4d0-2c94ba8bd946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.704000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76344-1fd3-4bae-b4d0-2c94ba8bd946 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.704308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76344-1fd3-4bae-b4d0-2c94ba8bd946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.704593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76344-1fd3-4bae-b4d0-2c94ba8bd946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.705417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d76344-1fd3-4bae-b4d0-2c94ba8bd946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.707250 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d76344-1fd3-4bae-b4d0-2c94ba8bd946 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.724075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1335/2658] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:39:20.725345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.738317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32784907-8a1b-4031-98a5-748e2931253f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:20.746672 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:20.748427 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.748882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.749357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] PATH_INFO: `/v3/projects/b108dd12cd004c41b8b85e49a312ff37/users/272f4ac56f384b4aa3102b3cd36fa3a5/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.769886 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b108dd12cd004c41b8b85e49a312ff37, user_id=272f4ac56f384b4aa3102b3cd36fa3a5, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:20.773132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:20.787994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1324/2659] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:20 2022] PUT /identity/v3/projects/b108dd12cd004c41b8b85e49a312ff37/users/272f4ac56f384b4aa3102b3cd36fa3a5/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:20.799821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:20.800119 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:20.800403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:20.831022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] MFA Rules not processed for user `272f4ac56f384b4aa3102b3cd36fa3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:20.832155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:20.907606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1336/2660] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:20.919849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:20.921765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:21.011231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb19177-9c84-4b6d-a3d6-bf61852e3ccc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:21.015809 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:21.016531 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:21.016858 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:21.017138 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:21.024824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:21.027423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:21.029272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:21.039879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1325/2661] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:21.618385 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:21.620129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:21.683593 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c4e15-aa26-409e-ac7f-4a8bf121e709 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:21.688127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:21.688896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:21.689182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:21.689458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:21.691560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:21.764504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:21.766156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:21.778236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1337/2662] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:24.836496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:24.838127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:24.935639 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08090054-c31b-4332-9458-7bc9119219ff service glance] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:24.958792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:24.959565 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:24.959890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:24.960780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:24.963481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:24.981307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:24.983188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:24.995073 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1326/2663] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:25.621903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:25.623655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:25.638869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:25.645686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388e653-2f7e-480c-ba39-be0964af645a service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:25.657094 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:25.657926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:25.658263 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:25.658505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] PATH_INFO: `/v3/users/9a0d49b3b50b4e668e157cb2e8e47da2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:25.663326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a0d49b3b50b4e668e157cb2e8e47da2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:25.663326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:25.731442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a0d49b3b50b4e668e157cb2e8e47da2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:25.743875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a0d49b3b50b4e668e157cb2e8e47da2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:25.759560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a0d49b3b50b4e668e157cb2e8e47da2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:25.760501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a0d49b3b50b4e668e157cb2e8e47da2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:25.762113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3ff36ff8-1933-4b82-ab37-bf295bface7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a0d49b3b50b4e668e157cb2e8e47da2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:25.785417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1338/2664] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:25 2022] DELETE /identity/v3/users/9a0d49b3b50b4e668e157cb2e8e47da2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:39:26.045889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.067654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de64599-91f7-4c1b-8f21-5b2601f8b375 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.142545 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.143453 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.143735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.144012 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] PATH_INFO: `/v3/projects/83cbbde066834f25a1b6f1bdc28e8b79` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.145314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=83cbbde066834f25a1b6f1bdc28e8b79)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.147377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.214391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83cbbde066834f25a1b6f1bdc28e8b79', 'request_id': 'req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:26.221569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83cbbde066834f25a1b6f1bdc28e8b79', 'request_id': 'req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:26.227635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83cbbde066834f25a1b6f1bdc28e8b79', 'request_id': 'req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:26.242850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] The token cache is being invalidate because project 83cbbde066834f25a1b6f1bdc28e8b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:26.243122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:26.243747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cedcfad1-77e8-4d3e-b0dc-b89114c5e33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:26.245612 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1327/2665] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:26 2022] DELETE /identity/v3/projects/83cbbde066834f25a1b6f1bdc28e8b79 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:26.251028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.251362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.251678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.273079 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:26.274321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:26.346932 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1339/2666] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:26.356421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f003ca-3093-497b-9635-a76934430515 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.357049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f003ca-3093-497b-9635-a76934430515 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.357616 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f003ca-3093-497b-9635-a76934430515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.388083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c3f003ca-3093-497b-9635-a76934430515 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:26.390306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c3f003ca-3093-497b-9635-a76934430515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:26.404469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1328/2667] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:26.408948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.420066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373a705-9ad6-4cb6-ac60-512f916cda3e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.426346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.427098 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.427379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.427650 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.428435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:39:26.428921 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.431251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.453715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1340/2668] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:39:26.456771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f003ca-3093-497b-9635-a76934430515 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.470635 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f003ca-3093-497b-9635-a76934430515 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.476624 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.477644 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.478102 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.478578 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.479775 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.483410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.491609 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:26.535606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1329/2669] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:26 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:26.537323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.559808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4ca07-9df0-433d-b9db-fc821c0d33f3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.636168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.637268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.637706 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.638165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.639494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.642303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.652976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:26.670200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1341/2670] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:26 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:26.675155 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.691552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9e2a5-2a5b-4c41-b979-b864d2e347a3 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.697486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c9463-d963-4817-839e-94192308226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.698174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9463-d963-4817-839e-94192308226b admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.698455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9463-d963-4817-839e-94192308226b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.698767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9463-d963-4817-839e-94192308226b admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.699517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c9463-d963-4817-839e-94192308226b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.701130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c9463-d963-4817-839e-94192308226b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.714666 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1330/2671] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:39:26.719467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.731295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b0d521-312a-43ba-8bec-b0a962e6b1ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.737643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb11bd9-681e-4989-b58d-72387a13839c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.738374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb11bd9-681e-4989-b58d-72387a13839c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.738734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb11bd9-681e-4989-b58d-72387a13839c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.739049 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb11bd9-681e-4989-b58d-72387a13839c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.739840 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb11bd9-681e-4989-b58d-72387a13839c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.741621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb11bd9-681e-4989-b58d-72387a13839c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.754832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1342/2672] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:26.759376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c9463-d963-4817-839e-94192308226b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.770355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c9463-d963-4817-839e-94192308226b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:26.777998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:26.780103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.781406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.781894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] PATH_INFO: `/v3/projects/46caece212e44751ba83b3dbfc8176ee/users/862f9a645a6f4ffe8c0518504ed6d47c/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.802500 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] RBAC: Authorizing `identity:create_grant(project_id=46caece212e44751ba83b3dbfc8176ee, user_id=862f9a645a6f4ffe8c0518504ed6d47c, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:26.805484 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:26.824873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1331/2673] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:26 2022] PUT /identity/v3/projects/46caece212e44751ba83b3dbfc8176ee/users/862f9a645a6f4ffe8c0518504ed6d47c/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:26.832728 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d941d-13b5-4c5a-985f-f41ccaa7009b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:26.833167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d941d-13b5-4c5a-985f-f41ccaa7009b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:26.833542 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d941d-13b5-4c5a-985f-f41ccaa7009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:26.859919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-919d941d-13b5-4c5a-985f-f41ccaa7009b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:26.861153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-919d941d-13b5-4c5a-985f-f41ccaa7009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:26.975396 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1343/2674] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:26.990001 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:26.992777 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:27.071781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5d1bd-93cd-4c42-a0c6-cf6fbe0e200c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:27.080222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:27.081637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:27.085898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:27.085898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:27.099455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:27.102192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:27.123353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1332/2675] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:39:32.672297 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b83d7-4c2e-45be-abc2-ae1f9fc9b872 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:32.673075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b83d7-4c2e-45be-abc2-ae1f9fc9b872 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:32.673927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b83d7-4c2e-45be-abc2-ae1f9fc9b872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:32.729159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4b5b83d7-4c2e-45be-abc2-ae1f9fc9b872 None None] MFA Rules not processed for user `862f9a645a6f4ffe8c0518504ed6d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:32.731872 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4b5b83d7-4c2e-45be-abc2-ae1f9fc9b872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:32.902102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1344/2676] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:32.919941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:32.921907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:33.048150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8073ffe4-e0b6-43a9-ad00-f111791c8191 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:33.058890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:33.060096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:33.060611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:33.061156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:33.076556 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:33.080501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:33.097965 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1333/2677] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:34.779901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:34.780735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:34.780735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:34.809112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] MFA Rules not processed for user `862f9a645a6f4ffe8c0518504ed6d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:34.811625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:34.827726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1345/2678] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:35.193792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:35.204508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2924cd-4103-4c13-8f57-872ed1d93323 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:35.208607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:35.209382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:35.209659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:35.209939 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:35.211951 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:35.224228 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:35.225683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ac665-b66b-495e-ae2e-52e42a63f864 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:35.235608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1334/2679] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:44.457462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:44.459506 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:44.475026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eae6f-006b-44f8-81cf-c53fca283a1d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:44.480824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:44.481743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:44.482076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:44.482392 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] PATH_INFO: `/v3/users/272f4ac56f384b4aa3102b3cd36fa3a5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:44.483696 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] RBAC: Authorizing `identity:delete_user(user_id=272f4ac56f384b4aa3102b3cd36fa3a5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:44.485969 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:44.553120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272f4ac56f384b4aa3102b3cd36fa3a5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:44.562054 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272f4ac56f384b4aa3102b3cd36fa3a5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:44.566804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:44.573279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272f4ac56f384b4aa3102b3cd36fa3a5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:44.574171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272f4ac56f384b4aa3102b3cd36fa3a5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:44.575120 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272f4ac56f384b4aa3102b3cd36fa3a5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:44.597370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1346/2680] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:44 2022] DELETE /identity/v3/users/272f4ac56f384b4aa3102b3cd36fa3a5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:44.602770 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:44.603343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:44.603626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:44.631363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:44.633115 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:44.665443 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:44.726178 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1335/2681] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:39:44.737658 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:44.739340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:44.802675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a81b8a-189c-4cd4-8de8-8c75f70a230a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:44.806484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822966b-cda1-4077-a71e-0b991422ded0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:44.807122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822966b-cda1-4077-a71e-0b991422ded0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:44.807578 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822966b-cda1-4077-a71e-0b991422ded0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:44.807913 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822966b-cda1-4077-a71e-0b991422ded0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:44.818102 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822966b-cda1-4077-a71e-0b991422ded0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:44.819594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2822966b-cda1-4077-a71e-0b991422ded0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:44.830064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1347/2682] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:45.145682 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.148021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:45.166355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213d12-7296-4774-8455-a2c0cc752d5c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.171592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.172471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.172752 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.173022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] PATH_INFO: `/v3/projects/b108dd12cd004c41b8b85e49a312ff37` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.174349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b108dd12cd004c41b8b85e49a312ff37)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.176364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.230960 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b108dd12cd004c41b8b85e49a312ff37', 'request_id': 'req-14cd6d56-de38-4229-a884-7fe9893f6b24'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:45.238618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b108dd12cd004c41b8b85e49a312ff37', 'request_id': 'req-14cd6d56-de38-4229-a884-7fe9893f6b24'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:45.244987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b108dd12cd004c41b8b85e49a312ff37', 'request_id': 'req-14cd6d56-de38-4229-a884-7fe9893f6b24'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:45.259859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] The token cache is being invalidate because project b108dd12cd004c41b8b85e49a312ff37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:45.260184 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:45.260805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-14cd6d56-de38-4229-a884-7fe9893f6b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:45.262390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1336/2683] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:45 2022] DELETE /identity/v3/projects/b108dd12cd004c41b8b85e49a312ff37 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:39:45.268624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.269170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.269303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.289345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:45.290675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:45.359982 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1348/2684] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:45.369315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.369766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.370094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.390719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:45.392259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:45.401079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1337/2685] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:45.405933 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.417464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e80b02-babb-44b1-a58b-90917ac9933f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.423281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.424163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.424641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.425043 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.425881 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:39:45.426289 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.428077 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.440882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1349/2686] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:39:45.446602 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.457210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99979c8-4ab0-440a-b1bd-625dc17fd8ad None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.462328 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.463426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.463608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.463995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.464425 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.466354 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.494275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1338/2687] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:45 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:45.498764 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.510341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e617a-e269-43d2-ad04-c02ec4d44672 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.572301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.572967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.573243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.573512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.574207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.576349 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.592238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:45.601280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1350/2688] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:45 2022] POST /identity/v3/users => generated 280 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:45.605969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.618167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a700d5b-b3b9-4586-9f6d-93a908291a26 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.623416 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.624157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.624475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.624789 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.625568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.627382 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.639539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1339/2689] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:39:45.644136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.654870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b08b8c-ed62-43d3-8923-09cf162496c6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.659900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dca76-515e-4690-b759-b48d7ddf4b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.660683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dca76-515e-4690-b759-b48d7ddf4b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.661063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dca76-515e-4690-b759-b48d7ddf4b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.661437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dca76-515e-4690-b759-b48d7ddf4b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.662272 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dca76-515e-4690-b759-b48d7ddf4b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.664164 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dca76-515e-4690-b759-b48d7ddf4b5c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.676295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1351/2690] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:45.681000 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.693076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455bdeac-2199-4d51-ad1f-e4138379d9cc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:45.698995 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:45.700126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.700437 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.700756 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] PATH_INFO: `/v3/projects/bff0796dd320409bbaf07c58f750c3f2/users/4773b76032b942ac8e54d1b5220c3889/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.720035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff0796dd320409bbaf07c58f750c3f2, user_id=4773b76032b942ac8e54d1b5220c3889, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:45.723042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:45.738318 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1340/2691] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:45 2022] PUT /identity/v3/projects/bff0796dd320409bbaf07c58f750c3f2/users/4773b76032b942ac8e54d1b5220c3889/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:45.806494 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:45.806798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:45.807084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:45.837687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] MFA Rules not processed for user `4773b76032b942ac8e54d1b5220c3889`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:45.839126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:45.909058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1352/2692] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:45 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:45.954143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:45.957848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:46.021798 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:46.029222 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721cea1-98b5-4437-a6a8-c62cc5194001 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:46.034839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33267dbe-bec0-41bc-bfcf-d3360dfc62e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:46.036074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33267dbe-bec0-41bc-bfcf-d3360dfc62e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:46.036593 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33267dbe-bec0-41bc-bfcf-d3360dfc62e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:46.036932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33267dbe-bec0-41bc-bfcf-d3360dfc62e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:46.048072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33267dbe-bec0-41bc-bfcf-d3360dfc62e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:46.050671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33267dbe-bec0-41bc-bfcf-d3360dfc62e2 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:46.061660 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1341/2693] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:45 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:46.078423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:46.177515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9e096-9558-454f-9506-c1dfc72a310f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:46.183016 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:46.184325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:46.184612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:46.184884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] PATH_INFO: `/v3/users/4773b76032b942ac8e54d1b5220c3889` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:46.186257 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4773b76032b942ac8e54d1b5220c3889)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:46.188264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:46.254022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4773b76032b942ac8e54d1b5220c3889'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:46.262463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4773b76032b942ac8e54d1b5220c3889'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:46.275974 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4773b76032b942ac8e54d1b5220c3889'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:46.276329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4773b76032b942ac8e54d1b5220c3889'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:46.276948 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4773b76032b942ac8e54d1b5220c3889'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:46.298477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:46.319621 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1353/2694] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:39:46 2022] DELETE /identity/v3/users/4773b76032b942ac8e54d1b5220c3889 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:46.325084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:46.325687 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:46.326209 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:46.358480 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:46.360355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:46.456673 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1342/2695] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:46.472559 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:46.475279 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:46.577863 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406757-f802-4be5-802c-7b8bdfbe27cf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:46.583171 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741994-0afd-46d3-98e2-2795bf11e8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:46.584103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741994-0afd-46d3-98e2-2795bf11e8af service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:46.584480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741994-0afd-46d3-98e2-2795bf11e8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:46.584866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741994-0afd-46d3-98e2-2795bf11e8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:46.598136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741994-0afd-46d3-98e2-2795bf11e8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:46.599836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3741994-0afd-46d3-98e2-2795bf11e8af service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:46.614800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1354/2696] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:39:47.016391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.019325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:47.034250 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e5e28-67ec-4b6d-a58d-dad9e72d7f0b None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.041064 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.041961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.042237 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.043235 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] PATH_INFO: `/v3/projects/bff0796dd320409bbaf07c58f750c3f2` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.044700 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff0796dd320409bbaf07c58f750c3f2)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.046707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.119166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff0796dd320409bbaf07c58f750c3f2', 'request_id': 'req-e6815d7d-4062-4ff3-9517-d50d7019b170'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:47.127010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff0796dd320409bbaf07c58f750c3f2', 'request_id': 'req-e6815d7d-4062-4ff3-9517-d50d7019b170'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:47.133447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff0796dd320409bbaf07c58f750c3f2', 'request_id': 'req-e6815d7d-4062-4ff3-9517-d50d7019b170'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:47.151641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] The token cache is being invalidate because project bff0796dd320409bbaf07c58f750c3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:39:47.151985 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:47.152908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-e6815d7d-4062-4ff3-9517-d50d7019b170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:39:47.155264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1343/2697] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:39:47 2022] DELETE /identity/v3/projects/bff0796dd320409bbaf07c58f750c3f2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:47.160980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.161268 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.161543 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.180311 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:47.181655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:47.279546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1355/2698] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:47.288633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.289049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.289218 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.312216 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:47.313461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:47.323754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1344/2699] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:39:47.328280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.340337 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc90b7a-eeba-4204-9b2b-be2f4c463010 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.348145 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.349435 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.349977 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.350454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.351640 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:39:47.356271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.356271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.373180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1356/2700] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:39:47.377971 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.388139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffaacf-a92e-4e9b-a680-b51cc5bb79dd None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.393605 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.394379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.394800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.395168 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.395997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.398035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.427059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1345/2701] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:47 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:39:47.431623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.442874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b51677-7afe-49f2-a0b7-359262746830 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.513464 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.514282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.514637 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.514957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.515804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.518020 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.547362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1357/2702] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:47 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:39:47.552871 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.568758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354954b8-27d6-4e27-b188-dbff8548b545 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.573897 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.574892 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.575217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.575497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.576243 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.577907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.594834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1346/2703] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:47.596345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.613726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d5604-7a48-4edf-8b02-45c41f840b1a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.619068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f7ddba-f7a8-40da-b12c-480d06cb4c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.622937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7ddba-f7a8-40da-b12c-480d06cb4c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.623369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7ddba-f7a8-40da-b12c-480d06cb4c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.623777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f7ddba-f7a8-40da-b12c-480d06cb4c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.624781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7ddba-f7a8-40da-b12c-480d06cb4c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.627815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f7ddba-f7a8-40da-b12c-480d06cb4c1d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.641063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1358/2704] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:39:47.647123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.660325 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aa8c9-367c-4f03-a2e3-560719aead2f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.666159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.667263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.667627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.667983 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] PATH_INFO: `/v3/projects/25f10e8177434ab69a22f1270cd8d0db/users/41e56871077745d18a51c2855a48cb7a/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.686928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25f10e8177434ab69a22f1270cd8d0db, user_id=41e56871077745d18a51c2855a48cb7a, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.690059 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.705527 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1347/2705] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:47 2022] PUT /identity/v3/projects/25f10e8177434ab69a22f1270cd8d0db/users/41e56871077745d18a51c2855a48cb7a/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:47.711868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36451ff9-b93b-4b23-8111-7453cd65f857 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.712243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36451ff9-b93b-4b23-8111-7453cd65f857 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.712588 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36451ff9-b93b-4b23-8111-7453cd65f857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.719897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:47.734084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-36451ff9-b93b-4b23-8111-7453cd65f857 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:47.735357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-36451ff9-b93b-4b23-8111-7453cd65f857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:47.808236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1359/2706] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:47.818992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:47.820933 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:47.833089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:47.890220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c298a0-cc8e-45ea-bc6d-61bcabb22b2e admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:47.894921 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:47.895637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:47.896003 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:47.896925 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:47.908049 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:47.909735 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:47.920767 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1348/2707] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:39:54.312505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:54.313103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:54.313103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:54.384457 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] MFA Rules not processed for user `41e56871077745d18a51c2855a48cb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:54.386504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:54.611622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1360/2708] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:54.616147 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:54.630369 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7b9f1-ea0f-4dae-ac76-08716c4c2e05 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:54.635719 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:54.636401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:54.636678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:54.637595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:54.637725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:54.640463 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:54.682498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1349/2709] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:39:54 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:39:54.687795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:54.721414 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a1ee2b-1b7e-48ff-ac77-1b01986d9c43 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:54.865002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:54.866216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:54.866838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:54.867238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:54.868106 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:54.870356 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:54.908231 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1361/2710] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:39:54 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:39:54.911800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:54.933907 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a5c8a-cbb0-4f94-b88e-701213969431 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:54.954116 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:54.955015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:54.955370 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:54.955720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:54.958082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:54.960998 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:54.988563 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1350/2711] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:54.989800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:55.015570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9574a7a-28a1-4c35-a417-bbb65c4492d0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:55.024682 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:55.025620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:55.025903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:55.026179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] PATH_INFO: `/v3/projects/04a6f647cbbb480eb41029883d189d80/users/bf66c72f1abd4e3e8dcbbbff05f9e216/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:55.049304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a6f647cbbb480eb41029883d189d80, user_id=bf66c72f1abd4e3e8dcbbbff05f9e216, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:55.055701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:55.080159 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:39:55.090751 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1362/2712] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:54 2022] PUT /identity/v3/projects/04a6f647cbbb480eb41029883d189d80/users/bf66c72f1abd4e3e8dcbbbff05f9e216/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:55.092157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:55.109934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa78a0-3f8b-48a7-a578-297ef87ca525 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:55.207889 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a649a9a-a816-4a6a-8e5b-75d986768e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:55.209341 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a649a9a-a816-4a6a-8e5b-75d986768e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:55.209873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a649a9a-a816-4a6a-8e5b-75d986768e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:55.210385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a649a9a-a816-4a6a-8e5b-75d986768e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:55.211717 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a649a9a-a816-4a6a-8e5b-75d986768e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:55.215895 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a649a9a-a816-4a6a-8e5b-75d986768e0c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:55.230969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1351/2713] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:39:55.235373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:55.248585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9cd23-5d3c-4a6b-8915-6f2528416c27 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:55.257801 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:55.258907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:55.259288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:55.259797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] PATH_INFO: `/v3/projects/04a6f647cbbb480eb41029883d189d80/users/bf66c72f1abd4e3e8dcbbbff05f9e216/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:55.261976 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a6f647cbbb480eb41029883d189d80, user_id=bf66c72f1abd4e3e8dcbbbff05f9e216, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:55.265067 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:55.279575 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1363/2714] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:39:55 2022] PUT /identity/v3/projects/04a6f647cbbb480eb41029883d189d80/users/bf66c72f1abd4e3e8dcbbbff05f9e216/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:39:55.804841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab8bb9-04d7-4d84-8f39-7716ad512eae None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:55.805621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab8bb9-04d7-4d84-8f39-7716ad512eae None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:55.805621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab8bb9-04d7-4d84-8f39-7716ad512eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:55.832090 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:39:55.840583 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-cdab8bb9-04d7-4d84-8f39-7716ad512eae None None] MFA Rules not processed for user `4453bd7126a849f29cfe3d8bdf4531f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:39:55.841711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:55.842917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-cdab8bb9-04d7-4d84-8f39-7716ad512eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:55.961231 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1352/2715] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:39:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:39:56.003369 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175468f-67fd-4c21-89f2-8098c8916d54 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:39:56.011876 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:39:56.014497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:39:56.014981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:39:56.015476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:39:56.018805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:39:56.059240 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:39:56.063798 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:39:56.079904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1364/2716] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:39:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:40:01.678863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e8f29-5029-421f-8239-07f82d9edc3c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:01.679390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e8f29-5029-421f-8239-07f82d9edc3c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:01.680834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e8f29-5029-421f-8239-07f82d9edc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:01.751210 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-981e8f29-5029-421f-8239-07f82d9edc3c None None] MFA Rules not processed for user `bf66c72f1abd4e3e8dcbbbff05f9e216`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:40:01.753025 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-981e8f29-5029-421f-8239-07f82d9edc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:40:01.774292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:40:01.907773 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1353/2717] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:40:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:40:01.918900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:40:01.921073 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:40:02.031765 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453b6c-7dce-4be2-80ae-930c939c6d9c service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:40:02.035826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fe7e2ad26c8483e8aecb89225f3b26e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:40:02.036558 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:02.037013 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:02.037300 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:02.136675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:40:02.139803 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:40:02.158835 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1365/2718] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:40:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:40:02.415525 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334686be-aa0e-4b11-90ae-a085452563cf None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:02.415911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334686be-aa0e-4b11-90ae-a085452563cf None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:02.416252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334686be-aa0e-4b11-90ae-a085452563cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:02.437778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-334686be-aa0e-4b11-90ae-a085452563cf None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:40:02.440625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-334686be-aa0e-4b11-90ae-a085452563cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:40:02.578574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1354/2719] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:40:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:40:02.645084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:40:02.648236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:40:02.736597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:40:02.796705 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6bafe-763a-419a-9cd6-d8859cd4977a service glance] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:40:02.806242 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808ca35-9cbf-4870-96aa-60c1aec94f0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:40:02.807794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ca35-9cbf-4870-96aa-60c1aec94f0f service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:02.808442 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ca35-9cbf-4870-96aa-60c1aec94f0f service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:02.808967 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808ca35-9cbf-4870-96aa-60c1aec94f0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:02.822541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808ca35-9cbf-4870-96aa-60c1aec94f0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:40:02.825342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808ca35-9cbf-4870-96aa-60c1aec94f0f service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:40:02.842797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1366/2720] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:40:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:40:12.551852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:12.552417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:12.552417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:12.572720 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] MFA Rules not processed for user `41e56871077745d18a51c2855a48cb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:40:12.573962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:40:12.582281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1355/2721] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:40:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:40:12.831726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=20 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:40:14.749069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:14.749711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:14.749711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:14.769743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:40:14.771316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:40:14.779775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1367/2722] 10.222.0.40 () {58 vars in 1009 bytes} [Fri Jul 15 15:40:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:40:14.795134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:40:14.810964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de05411-4e95-4a5f-83d7-ff16ea7d58ed None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:40:14.817941 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:40:14.818637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:14.819148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:14.819629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:14.830808 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:40:14.832786 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:40:14.846442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1356/2723] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:40:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:40:24.764551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:40:24.840522 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=59 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:40:49.345317 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:40:49.429866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba417-59eb-4b0c-994f-4bd269e37640 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:40:49.435966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75691ac0-88ab-4292-b427-9a7e13d6aa3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:40:49.437041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75691ac0-88ab-4292-b427-9a7e13d6aa3c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:40:49.437519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75691ac0-88ab-4292-b427-9a7e13d6aa3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:40:49.438002 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75691ac0-88ab-4292-b427-9a7e13d6aa3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:40:49.453628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75691ac0-88ab-4292-b427-9a7e13d6aa3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:40:49.455325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75691ac0-88ab-4292-b427-9a7e13d6aa3c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:40:49.465376 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1368/2724] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:40:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:40:59.460857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=26 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:41:36.686129 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:41:36.687969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:41:36.749279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b340d-7e6a-494f-98e7-a537691508bb service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:41:36.753042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:41:36.753769 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:41:36.754043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:41:36.754316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:41:36.756371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:41:36.771151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:41:36.772590 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:41:36.784119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1357/2725] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:41:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:41:45.819178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:41:45.819746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:41:45.819746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:41:45.840323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] MFA Rules not processed for user `862f9a645a6f4ffe8c0518504ed6d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:41:45.841557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:41:45.901278 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1369/2726] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:41:46.545904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:41:46.547378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:41:46.560028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392004-bfd5-49b3-926e-e7868f8b5d47 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:41:46.563949 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:41:46.564592 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:41:46.564874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:41:46.565167 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:41:46.567043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:41:46.582542 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:41:46.584185 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:41:46.598704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1358/2727] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:41:55.893684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:41:56.591418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=32 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:42:19.845170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:42:19.856459 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c291a-0457-4c61-8f31-240efc7677c1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:42:19.861009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:42:19.861693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:42:19.862113 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:42:19.862389 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:42:19.864520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:42:19.876495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:42:19.878440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb5120-d385-4354-8b8a-604a8a7f82a0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:42:19.888292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1370/2728] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:42:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:42:29.885036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:42:48.258430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:42:48.269253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9d04b-1c85-443f-bf29-0f5ee433ca85 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:42:48.273630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:42:48.274288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:42:48.274599 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:42:48.274878 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:42:48.277329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:42:48.289197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:42:48.290669 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:42:48.300574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1359/2729] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:42:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:42:58.295904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:43:00.415550 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:43:00.416141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:43:00.416141 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:43:00.435304 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] MFA Rules not processed for user `4453bd7126a849f29cfe3d8bdf4531f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:43:00.436601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:00.445166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1371/2730] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:43:00.801835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:43:00.813375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908032f-6619-4a9f-8fb0-ab2460a588bf service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:43:00.817999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:43:00.818662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:43:00.818954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:43:00.819236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:43:00.821385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:00.832447 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:43:00.833822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:43:00.842190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1360/2731] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:43:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:43:03.331366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:43:03.333123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:03.408636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8383-c41d-46b5-83fe-0efae711c9ed None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:43:03.414281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:43:03.414979 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:43:03.415229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:43:03.415524 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:43:03.417639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:03.430811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:43:03.432452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:43:03.443418 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1372/2732] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:43:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:43:08.985931 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:43:08.988930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:09.011614 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e89d3-7336-4da3-a425-8bb0e4b442d5 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:43:09.017844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:43:09.018852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:43:09.019223 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:43:09.019613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:43:09.023307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:09.038021 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:43:09.040372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:43:09.053594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1361/2733] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:43:13.438874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=35 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:43:19.047852 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=12 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:43:21.932944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:43:21.948778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51431ba-4c0e-485e-a378-b3de504ed98f service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:43:21.958689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:43:21.959357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:43:21.959643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:43:21.959949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:43:21.961941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:21.974792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:43:21.976390 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cc45a-ef60-4d6d-bff5-c8b4e2d02bd9 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:43:21.987796 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1373/2734] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:43:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:43:25.983620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:43:25.994842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fc5bb-fd78-4932-9d13-5945fda53c42 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:43:25.999551 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:43:26.000338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:43:26.000591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:43:26.000864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:43:26.002945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:43:26.071569 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:43:26.073004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:43:26.082739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1362/2735] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:43:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:43:31.983189 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:43:36.078558 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=30 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:08.973533 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b973de7-81e6-4fe0-810c-4d1dac50f507 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:08.974110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b973de7-81e6-4fe0-810c-4d1dac50f507 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:08.974110 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b973de7-81e6-4fe0-810c-4d1dac50f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:08.993712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4b973de7-81e6-4fe0-810c-4d1dac50f507 None None] MFA Rules not processed for user `791ca74ca97b4667a5c170f443982daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:08.994980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4b973de7-81e6-4fe0-810c-4d1dac50f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:09.002619 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1374/2736] 10.222.0.40 () {58 vars in 967 bytes} [Fri Jul 15 15:44:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 15:44:09.055291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:09.068849 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c712be1-dd50-43c0-ae11-6e4aae6993ef service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:09.073390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19b3f6ab1b924bb5907bbb33db1a5b4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:09.074060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:09.074342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:09.074641 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:09.084403 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:09.086867 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:09.098002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1363/2737] 10.222.0.40 () {60 vars in 1384 bytes} [Fri Jul 15 15:44:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 15:44:12.759144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:12.760104 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:12.760625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:12.787532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] MFA Rules not processed for user `862f9a645a6f4ffe8c0518504ed6d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:44:12.789363 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:12.798096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1375/2738] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:44:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:44:12.950100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:12.968944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483f25e-85de-4bea-8b19-01f0f3403545 service swift] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:12.984936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:12.986181 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:12.986765 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:12.987145 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:12.996636 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:13.011390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:13.013295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:13.026213 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1364/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:16.787665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:16.864161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b68f59-6928-4673-ba07-7bbee4ce5de4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:16.870287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:16.871502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:16.871772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:16.872052 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] PATH_INFO: `/v3/users/41e56871077745d18a51c2855a48cb7a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:16.873408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e56871077745d18a51c2855a48cb7a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:16.875518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:16.927672 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e56871077745d18a51c2855a48cb7a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:16.934698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e56871077745d18a51c2855a48cb7a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:16.945899 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e56871077745d18a51c2855a48cb7a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:16.946596 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e56871077745d18a51c2855a48cb7a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:16.947249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e56871077745d18a51c2855a48cb7a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:16.965879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1376/2740] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:44:16 2022] DELETE /identity/v3/users/41e56871077745d18a51c2855a48cb7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:44:17.149459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:17.165783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cd3f7-fe12-4542-ac4f-3d3485301037 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:17.232857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:17.233741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:17.234029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:17.234304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] PATH_INFO: `/v3/projects/25f10e8177434ab69a22f1270cd8d0db` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:17.235884 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25f10e8177434ab69a22f1270cd8d0db)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:17.237924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:17.294331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25f10e8177434ab69a22f1270cd8d0db', 'request_id': 'req-ba728686-c131-4a62-ac9e-b61ea364c2e6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.300831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25f10e8177434ab69a22f1270cd8d0db', 'request_id': 'req-ba728686-c131-4a62-ac9e-b61ea364c2e6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.307135 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25f10e8177434ab69a22f1270cd8d0db', 'request_id': 'req-ba728686-c131-4a62-ac9e-b61ea364c2e6'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.321932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] The token cache is being invalidate because project 25f10e8177434ab69a22f1270cd8d0db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:44:17.322502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.323187 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.324730 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1365/2741] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:44:17 2022] DELETE /identity/v3/projects/25f10e8177434ab69a22f1270cd8d0db => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:44:17.328792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:17.330235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:17.344288 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d83c46-6218-4edd-91c9-eeb2813542a3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:17.402903 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:17.403811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:17.404085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:17.404353 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] PATH_INFO: `/v3/users/bf66c72f1abd4e3e8dcbbbff05f9e216` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:17.405594 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf66c72f1abd4e3e8dcbbbff05f9e216)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:17.407556 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:17.414419 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:17.596643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf66c72f1abd4e3e8dcbbbff05f9e216'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.605200 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf66c72f1abd4e3e8dcbbbff05f9e216'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.615729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf66c72f1abd4e3e8dcbbbff05f9e216'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.616480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf66c72f1abd4e3e8dcbbbff05f9e216'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.617165 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf66c72f1abd4e3e8dcbbbff05f9e216'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.645137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1377/2742] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:44:17 2022] DELETE /identity/v3/users/bf66c72f1abd4e3e8dcbbbff05f9e216 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:44:17.832146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:17.834438 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:17.850029 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728686-c131-4a62-ac9e-b61ea364c2e6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:17.913160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:17.914030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:17.914316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:17.914642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] PATH_INFO: `/v3/projects/04a6f647cbbb480eb41029883d189d80` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:17.916036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a6f647cbbb480eb41029883d189d80)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:17.918019 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:17.924764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:17.978695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a6f647cbbb480eb41029883d189d80', 'request_id': 'req-ecf48203-64a8-4a50-b986-50189187ce7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.991092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a6f647cbbb480eb41029883d189d80', 'request_id': 'req-ecf48203-64a8-4a50-b986-50189187ce7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:17.998548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a6f647cbbb480eb41029883d189d80', 'request_id': 'req-ecf48203-64a8-4a50-b986-50189187ce7a'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:18.014454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] The token cache is being invalidate because project 04a6f647cbbb480eb41029883d189d80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:44:18.014814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:18.015493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:44:18.017076 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1366/2743] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:44:17 2022] DELETE /identity/v3/projects/04a6f647cbbb480eb41029883d189d80 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:44:27.638606 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:28.010697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:44:29.831723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:44:29.833484 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:29.991560 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd281d36-56fc-4525-aa79-bfa6a9fcf681 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:44:30.056785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:44:30.057364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:44:30.057647 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:44:30.057927 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:44:30.060203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:44:30.073121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:44:30.074776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:44:30.085772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1378/2744] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:44:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:44:40.080988 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:31.306360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:31.308459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:31.455560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf48203-64a8-4a50-b986-50189187ce7a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:31.462814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:31.463559 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:31.463845 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:31.464114 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:31.466044 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:31.480781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:31.481940 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:31.494415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1367/2745] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:45:41.491309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=45 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:42.675515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:42.676987 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:42.766544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfae43-f5ee-4c83-9e06-bec39bdd8536 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:42.771944 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:42.772592 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:42.772868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:42.773142 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:42.775235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:42.863896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:42.865666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:42.887483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1379/2746] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 15:45:45.882646 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:45.903479 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a45b03-af54-4eca-a1ed-dcde2dcbe173 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:45.910491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:45.911357 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:45.912276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:45.912276 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:45.914005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:45.992813 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:45.994836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:46.015538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1368/2747] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:46.506839 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:46.508287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:46.522695 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fec86-8b0e-4d83-845c-49c70493cdd6 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:46.527869 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:46.528553 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:46.528860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:46.529770 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:46.531874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:46.544681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:46.546313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:46.557761 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1380/2748] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:45:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:45:50.022383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:50.024254 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:50.042038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf3a91-b862-4371-b1b8-b673573da97c service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:50.046882 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:50.047779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:50.048065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:50.048340 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] PATH_INFO: `/v3/users/4453bd7126a849f29cfe3d8bdf4531f1` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:50.049589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4453bd7126a849f29cfe3d8bdf4531f1)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:50.052339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:50.107815 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4453bd7126a849f29cfe3d8bdf4531f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.119631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4453bd7126a849f29cfe3d8bdf4531f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.120481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4453bd7126a849f29cfe3d8bdf4531f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.121314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4453bd7126a849f29cfe3d8bdf4531f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.130304 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4453bd7126a849f29cfe3d8bdf4531f1'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.150097 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1369/2749] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:45:50 2022] DELETE /identity/v3/users/4453bd7126a849f29cfe3d8bdf4531f1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:45:50.342204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:50.353683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243db0a1-4868-491a-92de-a7ef99346715 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:50.402651 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:50.415683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:50.416544 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:50.416818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:50.417086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] PATH_INFO: `/v3/projects/9bda8b2837a54ec5a75c7dc355571255` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:50.418271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bda8b2837a54ec5a75c7dc355571255)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:50.420364 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:50.468797 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bda8b2837a54ec5a75c7dc355571255', 'request_id': 'req-a4a3f86f-ab50-445d-a432-903c70a02bce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.476114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bda8b2837a54ec5a75c7dc355571255', 'request_id': 'req-a4a3f86f-ab50-445d-a432-903c70a02bce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.483486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bda8b2837a54ec5a75c7dc355571255', 'request_id': 'req-a4a3f86f-ab50-445d-a432-903c70a02bce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.494331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] The token cache is being invalidate because project 9bda8b2837a54ec5a75c7dc355571255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:45:50.494645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.495283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.496804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1381/2750] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:45:50 2022] DELETE /identity/v3/projects/9bda8b2837a54ec5a75c7dc355571255 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:45:50.500821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:50.502233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:50.515329 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bc20-040e-4db9-a193-15b8d81e3894 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:50.572301 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:50.573084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:50.573362 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:50.573630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] PATH_INFO: `/v3/users/603bc3c0185346fa806d949c98133916` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:50.574924 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=603bc3c0185346fa806d949c98133916)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:50.576820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:50.591061 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:45:50.628754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '603bc3c0185346fa806d949c98133916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.636623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '603bc3c0185346fa806d949c98133916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.637366 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '603bc3c0185346fa806d949c98133916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.638042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '603bc3c0185346fa806d949c98133916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.646671 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-cadb5705-96a0-4f1c-b1cf-94bc5948dff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '603bc3c0185346fa806d949c98133916'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:50.667174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1370/2751] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:45:50 2022] DELETE /identity/v3/users/603bc3c0185346fa806d949c98133916 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:45:50.861949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:45:50.864041 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:45:50.886097 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3f86f-ab50-445d-a432-903c70a02bce admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:45:50.965552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:45:50.966443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:45:50.967065 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:45:50.967656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] PATH_INFO: `/v3/projects/e92a81048fda4c9e91da130fb6135fb8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:45:50.968884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e92a81048fda4c9e91da130fb6135fb8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:45:50.972256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:45:51.031452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e92a81048fda4c9e91da130fb6135fb8', 'request_id': 'req-26afd3c8-808f-43d4-830c-904c91c8bb1d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:51.043177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e92a81048fda4c9e91da130fb6135fb8', 'request_id': 'req-26afd3c8-808f-43d4-830c-904c91c8bb1d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:51.051841 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e92a81048fda4c9e91da130fb6135fb8', 'request_id': 'req-26afd3c8-808f-43d4-830c-904c91c8bb1d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:51.067671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] The token cache is being invalidate because project e92a81048fda4c9e91da130fb6135fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:45:51.068047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:51.068659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:45:51.070582 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1382/2752] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:45:50 2022] DELETE /identity/v3/projects/e92a81048fda4c9e91da130fb6135fb8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:46:00.663292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:01.063276 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:02.810350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:02.810955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:02.811127 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:02.832084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] MFA Rules not processed for user `862f9a645a6f4ffe8c0518504ed6d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:46:02.833234 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:02.909090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1371/2753] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:46:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:46:03.258324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:46:03.260063 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:03.318723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afd3c8-808f-43d4-830c-904c91c8bb1d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:46:03.323915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:46:03.324570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:46:03.324852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:46:03.325126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:46:03.327211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:46:03.337732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:46:03.339223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:46:03.347326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1383/2754] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:46:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:46:12.901176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=25 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:46:13.343808 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=26 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:33.189236 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:33.200528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f7366-f298-4700-ade1-2f077949a8f3 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:33.205040 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:33.205885 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:33.206166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:33.207094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:33.209131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:33.221553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:33.223142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:33.232060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1372/2755] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:34.449962 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:34.451576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:34.535168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc9304-b16f-4458-8cd3-bcadf9d946bf service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:34.539711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:34.540361 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:34.540638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:34.540888 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:34.542793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:34.637187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:34.638579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:34.661429 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1384/2756] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:42.568071 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:42.569637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:42.583112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e043-e17c-4a48-aab0-d4d8fde5a5d8 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:42.587056 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:42.587732 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:42.588011 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:42.588286 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:42.591060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:42.607206 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:42.608751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:42.621977 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1373/2757] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:44.658168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=75 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:52.211480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:52.222100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6fc689-5a6d-4588-9d70-1e6168f9f0eb service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:52.226960 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:52.227622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:52.227882 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:52.228166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:52.230674 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:52.313344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:52.315176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:52.328056 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1385/2758] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:52.615458 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=12 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:55.735865 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:55.737793 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:55.751744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bb49f-65cd-4e4e-afd2-89e0ed7bfec1 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:55.756408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:55.757291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:55.757570 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:55.757839 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] PATH_INFO: `/v3/users/862f9a645a6f4ffe8c0518504ed6d47c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:55.759074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=862f9a645a6f4ffe8c0518504ed6d47c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:55.761131 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:55.830402 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '862f9a645a6f4ffe8c0518504ed6d47c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:55.841387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '862f9a645a6f4ffe8c0518504ed6d47c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:55.842495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '862f9a645a6f4ffe8c0518504ed6d47c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:55.843384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '862f9a645a6f4ffe8c0518504ed6d47c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:55.856818 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-759084a6-39e1-47e6-8ed2-c7c65cf56a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '862f9a645a6f4ffe8c0518504ed6d47c'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:55.877295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1374/2759] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:47:55 2022] DELETE /identity/v3/users/862f9a645a6f4ffe8c0518504ed6d47c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:47:56.089038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.100201 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede11709-0b73-4527-9921-f34f6ef39c9f service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.168526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.169398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.169679 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.169950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] PATH_INFO: `/v3/projects/46caece212e44751ba83b3dbfc8176ee` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.171182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46caece212e44751ba83b3dbfc8176ee)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.173154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.225576 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46caece212e44751ba83b3dbfc8176ee', 'request_id': 'req-ceef37f2-caf9-4f04-a58e-1910c088c2f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:56.235865 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46caece212e44751ba83b3dbfc8176ee', 'request_id': 'req-ceef37f2-caf9-4f04-a58e-1910c088c2f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:56.242986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46caece212e44751ba83b3dbfc8176ee', 'request_id': 'req-ceef37f2-caf9-4f04-a58e-1910c088c2f7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:56.255643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] The token cache is being invalidate because project 46caece212e44751ba83b3dbfc8176ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:47:56.255643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:56.256526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ceef37f2-caf9-4f04-a58e-1910c088c2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:47:56.258918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1386/2760] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:47:56 2022] DELETE /identity/v3/projects/46caece212e44751ba83b3dbfc8176ee => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:47:56.265182 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.265435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.265725 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.329280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:56.330695 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:56.394372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1375/2761] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:56.403492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.403732 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.404160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.429539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:56.433586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:56.440642 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1387/2762] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:56.445401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.456539 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673842b-5a40-47ec-b024-836ba00f9eba None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.462611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.463267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.463562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.463830 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.464831 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:47:56.465104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.466714 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.479961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1376/2763] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:47:56.484769 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.495105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457aeca-110b-46db-a68f-d038b43a9b9a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.500713 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.501373 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.501648 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.501917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.502660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.504557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.519827 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:56.544699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1388/2764] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:47:56 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:47:56.549215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.560501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4310935-0a2e-4733-9d07-cb0c710f844c admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.589010 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:56.629561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.630375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.630699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.630982 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.631686 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.633621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.655173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1377/2765] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:47:56 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:47:56.660604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.671917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318f7e9-bf3f-4994-8080-6edbf23bb837 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.677108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.677774 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.678055 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.678331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.679101 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.680750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.691780 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1389/2766] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:47:56.696580 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.708224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c8255-ae39-43ee-9df3-7079f28a6347 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.713509 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67387006-bbb7-49fe-bcd8-fb8e09269895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.714386 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387006-bbb7-49fe-bcd8-fb8e09269895 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.714744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387006-bbb7-49fe-bcd8-fb8e09269895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.715078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387006-bbb7-49fe-bcd8-fb8e09269895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.715877 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67387006-bbb7-49fe-bcd8-fb8e09269895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.717548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67387006-bbb7-49fe-bcd8-fb8e09269895 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.730573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1378/2767] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:47:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:47:56.735023 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.747789 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fd4e5-4770-4ab2-be6c-55167202aa7a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.754027 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.755126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.755477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.755823 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] PATH_INFO: `/v3/projects/16f5aff8178c4e039240e04714d23254/users/4530e1bd57d3432a800f74890608c36c/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.773229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f5aff8178c4e039240e04714d23254, user_id=4530e1bd57d3432a800f74890608c36c, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:56.776155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:56.791519 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1390/2768] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:47:56 2022] PUT /identity/v3/projects/16f5aff8178c4e039240e04714d23254/users/4530e1bd57d3432a800f74890608c36c/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:47:56.802888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:56.803385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:56.803475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:56.834638 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] MFA Rules not processed for user `4530e1bd57d3432a800f74890608c36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:56.835896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:56.916433 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1379/2769] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:47:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:56.921752 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:56.993303 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a6f40-ddf6-427b-87ee-36446f84c318 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:56.999154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:56.999846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.000131 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.000407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.001138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.003428 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.032851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1391/2770] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:47:56 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:47:57.037890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.052196 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691167d0-8120-4f1b-a96b-94da152b455f None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.119435 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.120105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.120387 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.120659 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.121807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.123454 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.147859 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1380/2771] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:47:57 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:47:57.152323 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.164220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba95e2-d06f-4631-9bc7-84d3c092477f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.169845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.170548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.170848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.171122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.171851 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.173512 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.185095 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1392/2772] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:47:57.189633 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.201733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd790-63f8-41cf-a16d-4aa43288e7c1 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.207077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cef37c-ba1a-4c3d-98b7-bfc234e095a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.207766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cef37c-ba1a-4c3d-98b7-bfc234e095a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.208046 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cef37c-ba1a-4c3d-98b7-bfc234e095a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.208332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cef37c-ba1a-4c3d-98b7-bfc234e095a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.209094 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cef37c-ba1a-4c3d-98b7-bfc234e095a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.210874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cef37c-ba1a-4c3d-98b7-bfc234e095a5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.222915 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1381/2773] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:47:57.227433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.240109 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a7e29-3fa7-4292-a747-86d54db5cc06 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.247127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.248354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.248635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.248917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] PATH_INFO: `/v3/projects/55e5556cd4dc4852b11994f1552fa74f/users/05e4cfd809ab43618cc39289d8fbee34/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.268811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e5556cd4dc4852b11994f1552fa74f, user_id=05e4cfd809ab43618cc39289d8fbee34, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.271612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.286115 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1393/2774] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:47:57 2022] PUT /identity/v3/projects/55e5556cd4dc4852b11994f1552fa74f/users/05e4cfd809ab43618cc39289d8fbee34/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:47:57.298047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.298339 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.298650 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.330560 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] MFA Rules not processed for user `05e4cfd809ab43618cc39289d8fbee34`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:57.331835 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:57.336573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:57.410009 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1382/2775] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:57.416328 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.493939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74983b-4cf5-41c6-86e2-b5c08cca489b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.500788 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.501387 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.501659 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.503846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.503846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.504654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.509671 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:57.534891 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1394/2776] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:47:57 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:47:57.539363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.552457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372803-eecf-4ac8-8e1e-b22c655ff401 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.622353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.623087 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.623390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.623648 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.624363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.626298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.652316 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1383/2777] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:47:57 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:47:57.654381 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.666964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ea661-925a-48ec-8f9b-1f43d3be4746 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.672985 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.673656 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.673936 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.674204 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.674963 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.676590 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.689084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1395/2778] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:47:57.694015 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.705774 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fad3f-c947-41dc-b064-3fcda2b92ca5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.711543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.712455 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.712739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.713020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] PATH_INFO: `/v3/projects/cef8597a6e4547889fc681ec0a36bf47/users/291be39e3026444db009e08cd7ed7c67/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.730027 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef8597a6e4547889fc681ec0a36bf47, user_id=291be39e3026444db009e08cd7ed7c67, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.733372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.751330 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1384/2779] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:47:57 2022] PUT /identity/v3/projects/cef8597a6e4547889fc681ec0a36bf47/users/291be39e3026444db009e08cd7ed7c67/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:47:57.755378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.766139 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b2bb2-960e-4c18-a308-5a1e7cf5c95b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.827523 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a8627-bb19-4cbc-b73d-0fb0d8371c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.828170 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a8627-bb19-4cbc-b73d-0fb0d8371c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.828481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a8627-bb19-4cbc-b73d-0fb0d8371c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.828746 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a8627-bb19-4cbc-b73d-0fb0d8371c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.829601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a8627-bb19-4cbc-b73d-0fb0d8371c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.831128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a8627-bb19-4cbc-b73d-0fb0d8371c59 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.842342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1396/2780] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:47:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:47:57.847060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:57.859092 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb33e6-07f6-4f6f-a7d4-63cbfe1285c4 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:57.864348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:57.865266 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.865543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.865810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] PATH_INFO: `/v3/projects/cef8597a6e4547889fc681ec0a36bf47/users/291be39e3026444db009e08cd7ed7c67/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.867954 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef8597a6e4547889fc681ec0a36bf47, user_id=291be39e3026444db009e08cd7ed7c67, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:57.871289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:57.883156 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1385/2781] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:47:57 2022] PUT /identity/v3/projects/cef8597a6e4547889fc681ec0a36bf47/users/291be39e3026444db009e08cd7ed7c67/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:47:57.894630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:57.894964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:57.895408 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:57.925907 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] MFA Rules not processed for user `291be39e3026444db009e08cd7ed7c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:47:57.927154 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:58.004319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1397/2782] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:47:58.016186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:58.017936 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:58.076942 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad98c3a-f1f8-4de5-94da-e94f65ddffaf admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:58.081153 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:58.081800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:58.082077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:58.082348 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:58.092451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:58.093887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:58.099078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:58.104498 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1386/2783] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:47:58.686193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:47:58.687736 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:47:58.701557 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe25121-7c57-4df4-b881-8d36bfa2e3d2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:47:58.705664 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:47:58.706324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:47:58.706630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:47:58.706904 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:47:58.772714 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:47:58.775561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:47:58.777562 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:47:58.789123 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1398/2784] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:47:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:02.871423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:02.881917 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17baa6-68d6-4bbc-9975-78141bcb0196 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:02.885980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84182630-7da2-4291-a08e-6ac68c80050a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:02.886711 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84182630-7da2-4291-a08e-6ac68c80050a service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:02.886991 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84182630-7da2-4291-a08e-6ac68c80050a service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:02.887267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84182630-7da2-4291-a08e-6ac68c80050a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:02.951822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84182630-7da2-4291-a08e-6ac68c80050a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:02.953311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84182630-7da2-4291-a08e-6ac68c80050a service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:02.962838 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1387/2785] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:05.154071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:05.223566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42233e7-7f1f-49ab-af86-1167ecc9877b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:05.228211 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:05.229091 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:05.229362 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:05.229750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] PATH_INFO: `/v3/users/4530e1bd57d3432a800f74890608c36c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:05.231181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4530e1bd57d3432a800f74890608c36c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:05.233125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:05.289624 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4530e1bd57d3432a800f74890608c36c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.296607 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4530e1bd57d3432a800f74890608c36c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.305595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4530e1bd57d3432a800f74890608c36c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.306215 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4530e1bd57d3432a800f74890608c36c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.306844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4530e1bd57d3432a800f74890608c36c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.332286 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1399/2786] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:05 2022] DELETE /identity/v3/users/4530e1bd57d3432a800f74890608c36c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:05.337215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:05.337723 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:05.337844 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:05.359409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:05.360621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:05.433794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1388/2787] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:05.444152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:05.445762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:05.504836 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de94163-9458-4df6-b4b5-1d10f992fa96 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:05.509340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:05.509980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:05.510262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:05.510601 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:05.520750 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:05.522207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:05.532322 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1400/2788] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:05.847299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:05.848861 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:05.871338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5808640-4c2b-4917-93f0-8655a4d2f9a0 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:05.876452 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:05.877401 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:05.877746 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:05.878022 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] PATH_INFO: `/v3/projects/16f5aff8178c4e039240e04714d23254` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:05.879249 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f5aff8178c4e039240e04714d23254)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:05.881183 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:05.935899 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f5aff8178c4e039240e04714d23254', 'request_id': 'req-1e77eebb-fc88-41c5-9070-090de2d5a651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.942594 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f5aff8178c4e039240e04714d23254', 'request_id': 'req-1e77eebb-fc88-41c5-9070-090de2d5a651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.948406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f5aff8178c4e039240e04714d23254', 'request_id': 'req-1e77eebb-fc88-41c5-9070-090de2d5a651'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.967950 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] The token cache is being invalidate because project 16f5aff8178c4e039240e04714d23254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:05.968394 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.969350 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:05.971162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1389/2789] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:05 2022] DELETE /identity/v3/projects/16f5aff8178c4e039240e04714d23254 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:05.975508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:05.985955 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f69888-42cd-4be1-83b2-5041bb76427b service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:06.000384 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:06.058163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:06.059108 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:06.059405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:06.059698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] PATH_INFO: `/v3/users/05e4cfd809ab43618cc39289d8fbee34` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:06.060943 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e4cfd809ab43618cc39289d8fbee34)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:06.062957 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:06.113285 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e4cfd809ab43618cc39289d8fbee34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.121412 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e4cfd809ab43618cc39289d8fbee34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.130635 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e4cfd809ab43618cc39289d8fbee34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.131360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e4cfd809ab43618cc39289d8fbee34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.132105 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e4cfd809ab43618cc39289d8fbee34'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.151430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1401/2790] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:05 2022] DELETE /identity/v3/users/05e4cfd809ab43618cc39289d8fbee34 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:06.447604 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:06.449157 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:06.464955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77eebb-fc88-41c5-9070-090de2d5a651 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:06.487828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:06.532150 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:06.533039 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:06.533317 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:06.533601 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] PATH_INFO: `/v3/projects/55e5556cd4dc4852b11994f1552fa74f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:06.535618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e5556cd4dc4852b11994f1552fa74f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:06.538376 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:06.591521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e5556cd4dc4852b11994f1552fa74f', 'request_id': 'req-03f039eb-e145-48b4-b8da-8f33234316a2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.597724 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e5556cd4dc4852b11994f1552fa74f', 'request_id': 'req-03f039eb-e145-48b4-b8da-8f33234316a2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.605060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e5556cd4dc4852b11994f1552fa74f', 'request_id': 'req-03f039eb-e145-48b4-b8da-8f33234316a2'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.620698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] The token cache is being invalidate because project 55e5556cd4dc4852b11994f1552fa74f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:06.620981 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.621819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.623571 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1390/2791] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:06 2022] DELETE /identity/v3/projects/55e5556cd4dc4852b11994f1552fa74f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:06.629526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:06.631388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:06.645326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce21d-e225-4d45-9a4c-6e338b9123bb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:06.712938 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:06.713826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:06.714103 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:06.714370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] PATH_INFO: `/v3/users/291be39e3026444db009e08cd7ed7c67` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:06.715591 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291be39e3026444db009e08cd7ed7c67)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:06.717539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:06.763730 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:06.772153 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291be39e3026444db009e08cd7ed7c67'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.781711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291be39e3026444db009e08cd7ed7c67'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.790715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291be39e3026444db009e08cd7ed7c67'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.791628 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291be39e3026444db009e08cd7ed7c67'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.792035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5f1d983-11ac-4281-9e59-7c2e91bc2225 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291be39e3026444db009e08cd7ed7c67'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:06.811567 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1402/2792] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:06 2022] DELETE /identity/v3/users/291be39e3026444db009e08cd7ed7c67 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:07.116914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.125293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:07.138620 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f039eb-e145-48b4-b8da-8f33234316a2 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.206351 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.207626 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.207964 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.208406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] PATH_INFO: `/v3/projects/cef8597a6e4547889fc681ec0a36bf47` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.210158 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef8597a6e4547889fc681ec0a36bf47)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.213347 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.274406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef8597a6e4547889fc681ec0a36bf47', 'request_id': 'req-280281f6-8b76-4ab4-91a3-1d3faf220141'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:07.281477 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef8597a6e4547889fc681ec0a36bf47', 'request_id': 'req-280281f6-8b76-4ab4-91a3-1d3faf220141'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:07.282759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:07.289263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef8597a6e4547889fc681ec0a36bf47', 'request_id': 'req-280281f6-8b76-4ab4-91a3-1d3faf220141'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:07.303202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] The token cache is being invalidate because project cef8597a6e4547889fc681ec0a36bf47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:07.303478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:07.304128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-280281f6-8b76-4ab4-91a3-1d3faf220141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:07.305797 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1391/2793] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:07 2022] DELETE /identity/v3/projects/cef8597a6e4547889fc681ec0a36bf47 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:07.312138 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.312426 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.312727 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.332340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:07.333569 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:07.399684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1403/2794] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:07.409974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.410299 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.410547 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.430828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:07.432154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:07.441679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1392/2795] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:07.446229 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.458034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6946e-95d7-4d0f-b4a6-a8f8c44839cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.471352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.472069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.472355 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.472634 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.473377 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:48:07.473686 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.475360 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.488262 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1404/2796] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:48:07.493449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.503461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f461a95-bab5-4cc3-87f1-c6df139d47ef None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.508349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.509008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.509355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.509630 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.510345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.512326 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.539811 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1393/2797] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:07 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:07.544793 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.555959 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293b04-de8e-46d8-951f-6995762b06db admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.624502 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.625254 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.625538 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.625856 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.626579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.628577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.654645 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1405/2798] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:07 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:07.659005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.676269 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c0aa8-e425-4483-94b9-0571b860030b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.682507 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.683220 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.683491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.683934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.684771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.686407 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.697872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1394/2799] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:07.702223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.712864 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123737af-b1cc-4bb8-b71e-17037784493b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.719126 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea77fd-917f-430b-afda-72485f6616e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.719812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea77fd-917f-430b-afda-72485f6616e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.720099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea77fd-917f-430b-afda-72485f6616e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.720378 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea77fd-917f-430b-afda-72485f6616e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.721121 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea77fd-917f-430b-afda-72485f6616e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.722783 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea77fd-917f-430b-afda-72485f6616e1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.734781 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1406/2800] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:07.739561 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:07.752973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c303c-1d0c-4c98-9416-ad5caeea48ce admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:07.760754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:07.761668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.762245 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.762654 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] PATH_INFO: `/v3/projects/cfdb50e458034afba298ee95ea482a8d/users/e57e08f426f7463c86681823a9a2902d/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.782295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdb50e458034afba298ee95ea482a8d, user_id=e57e08f426f7463c86681823a9a2902d, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:07.785047 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:07.801166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1395/2801] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:07 2022] PUT /identity/v3/projects/cfdb50e458034afba298ee95ea482a8d/users/e57e08f426f7463c86681823a9a2902d/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:07.813525 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:07.813794 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:07.814072 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:07.844815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] MFA Rules not processed for user `e57e08f426f7463c86681823a9a2902d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:07.846046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:07.857046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:07.922423 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1407/2802] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:07.929448 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.001050 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42053bfe-cc61-4284-aacd-4b42e196195f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.007020 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.007667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.007970 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.008239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.008927 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.010988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.043002 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1396/2803] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:07 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:08.047675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.059190 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf10e0-3aa7-4d54-96a4-60270810d093 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.129886 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.131483 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.132007 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.132397 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.133458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.135487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.167917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1408/2804] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:08 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:08.172375 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.184946 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9c0d0-c198-4ec1-8096-ebabb9fab3b6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.192306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.193288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.193745 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.194177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.195365 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.197072 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.209794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1397/2805] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:08.214723 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.225460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132af4c-a4b3-41be-81e6-f8d0dd55cc2a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.230772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.231837 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.232180 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.232438 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] PATH_INFO: `/v3/projects/7c7f057a7d18462e8a53ee3488ca5b08/users/db61ce0056eb4754bae39fd311790a74/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.252345 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7f057a7d18462e8a53ee3488ca5b08, user_id=db61ce0056eb4754bae39fd311790a74, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.255069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.270633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1409/2806] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:08 2022] PUT /identity/v3/projects/7c7f057a7d18462e8a53ee3488ca5b08/users/db61ce0056eb4754bae39fd311790a74/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:08.274819 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.286164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dd636-a9c2-46b4-b2bc-b56390e40433 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.346856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:08.363083 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe46f40-29d2-43b8-975a-5005325ee879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.363749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe46f40-29d2-43b8-975a-5005325ee879 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.364035 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe46f40-29d2-43b8-975a-5005325ee879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.364334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe46f40-29d2-43b8-975a-5005325ee879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.365066 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe46f40-29d2-43b8-975a-5005325ee879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.366716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe46f40-29d2-43b8-975a-5005325ee879 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.379631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1398/2807] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:08 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:08.384858 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.396430 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cd392-0426-4605-a85c-9e5377b70218 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.402298 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.403589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.403885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.404143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] PATH_INFO: `/v3/projects/7c7f057a7d18462e8a53ee3488ca5b08/users/db61ce0056eb4754bae39fd311790a74/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.406319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7f057a7d18462e8a53ee3488ca5b08, user_id=db61ce0056eb4754bae39fd311790a74, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.409167 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.423467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1410/2808] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:08 2022] PUT /identity/v3/projects/7c7f057a7d18462e8a53ee3488ca5b08/users/db61ce0056eb4754bae39fd311790a74/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:08.435766 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.436070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.436343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.465374 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] MFA Rules not processed for user `db61ce0056eb4754bae39fd311790a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:08.466621 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:08.550908 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1399/2809] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:08.562342 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.564336 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:08.637188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7fe07-23bb-4030-81ab-e1f12e52a5b5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.641252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.641930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.642212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.642481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.652467 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.653994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.664114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1411/2810] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:08.769123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:08.784728 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c7b1c1-0166-4a73-b48c-bd8a6f932862 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:08.788930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6067e61-6ed6-42d6-9b08-8b05838ee341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:08.789586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6067e61-6ed6-42d6-9b08-8b05838ee341 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:08.789862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6067e61-6ed6-42d6-9b08-8b05838ee341 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:08.790128 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6067e61-6ed6-42d6-9b08-8b05838ee341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:08.858825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6067e61-6ed6-42d6-9b08-8b05838ee341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:08.860302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6067e61-6ed6-42d6-9b08-8b05838ee341 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:08.874291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1400/2811] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:11.504685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:11.514508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:11.574949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd9987-a08d-48f8-8c04-e267288ebbd3 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:11.580178 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:11.581076 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:11.581354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:11.581626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] PATH_INFO: `/v3/users/e57e08f426f7463c86681823a9a2902d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:11.583133 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e57e08f426f7463c86681823a9a2902d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:11.585093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:11.637900 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e57e08f426f7463c86681823a9a2902d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:11.645301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e57e08f426f7463c86681823a9a2902d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:11.654946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e57e08f426f7463c86681823a9a2902d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:11.655566 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e57e08f426f7463c86681823a9a2902d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:11.656284 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e57e08f426f7463c86681823a9a2902d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:11.674896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1412/2812] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:11 2022] DELETE /identity/v3/users/e57e08f426f7463c86681823a9a2902d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:11.679750 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:11.680261 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:11.680363 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:11.701221 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:11.702469 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:11.764690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1401/2813] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:11.774689 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:11.776270 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:11.839030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9e707-aff2-4f67-9d61-721d0e6e7084 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:11.843324 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:11.843980 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:11.844260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:11.844526 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:11.854193 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:11.855757 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:11.865527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1413/2814] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:12.141651 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:12.143257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:12.155857 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b8bfe9-bf49-40a0-93e0-f7d68a8b798c None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:12.160627 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:12.161506 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:12.161780 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:12.162042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] PATH_INFO: `/v3/projects/cfdb50e458034afba298ee95ea482a8d` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:12.163291 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfdb50e458034afba298ee95ea482a8d)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:12.165292 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:12.174988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=100 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:12.217412 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfdb50e458034afba298ee95ea482a8d', 'request_id': 'req-665be4f0-32e7-47bb-9a8f-4422938b869f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.225613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfdb50e458034afba298ee95ea482a8d', 'request_id': 'req-665be4f0-32e7-47bb-9a8f-4422938b869f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.233485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfdb50e458034afba298ee95ea482a8d', 'request_id': 'req-665be4f0-32e7-47bb-9a8f-4422938b869f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.250132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] The token cache is being invalidate because project cfdb50e458034afba298ee95ea482a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:12.250862 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.251973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.253529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1402/2815] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:12 2022] DELETE /identity/v3/projects/cfdb50e458034afba298ee95ea482a8d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:12.258316 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:12.270039 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da029b-5bc3-477f-bb69-77658871f8bb service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:12.298332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:12.332573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:12.333475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:12.333753 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:12.334021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] PATH_INFO: `/v3/users/db61ce0056eb4754bae39fd311790a74` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:12.335306 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db61ce0056eb4754bae39fd311790a74)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:12.337241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:12.382889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db61ce0056eb4754bae39fd311790a74'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.390034 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db61ce0056eb4754bae39fd311790a74'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.398909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db61ce0056eb4754bae39fd311790a74'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.399577 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db61ce0056eb4754bae39fd311790a74'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.400210 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89c35feb-2a62-4f69-9a00-64901f24d012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db61ce0056eb4754bae39fd311790a74'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.424522 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1414/2816] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:12 2022] DELETE /identity/v3/users/db61ce0056eb4754bae39fd311790a74 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:48:12.726619 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:12.728622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:12.742764 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665be4f0-32e7-47bb-9a8f-4422938b869f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:12.804690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:12.805470 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:12.805753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:12.806030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] PATH_INFO: `/v3/projects/7c7f057a7d18462e8a53ee3488ca5b08` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:12.807302 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c7f057a7d18462e8a53ee3488ca5b08)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:12.809696 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:12.863822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c7f057a7d18462e8a53ee3488ca5b08', 'request_id': 'req-9352f990-61e4-47be-879a-3bfc8366f720'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.870397 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c7f057a7d18462e8a53ee3488ca5b08', 'request_id': 'req-9352f990-61e4-47be-879a-3bfc8366f720'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.876783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c7f057a7d18462e8a53ee3488ca5b08', 'request_id': 'req-9352f990-61e4-47be-879a-3bfc8366f720'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.890201 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] The token cache is being invalidate because project 7c7f057a7d18462e8a53ee3488ca5b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:12.890491 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.891343 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-9352f990-61e4-47be-879a-3bfc8366f720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:12.893038 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1403/2817] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:12 2022] DELETE /identity/v3/projects/7c7f057a7d18462e8a53ee3488ca5b08 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:12.898069 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:12.898603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:12.898717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:12.918326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:12.919579 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:12.982986 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1415/2818] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:13.046074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b2ab-f425-469a-b51e-596414260127 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.046385 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b2ab-f425-469a-b51e-596414260127 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.046685 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248b2ab-f425-469a-b51e-596414260127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.066165 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-7248b2ab-f425-469a-b51e-596414260127 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:13.067371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-7248b2ab-f425-469a-b51e-596414260127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:13.077481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1404/2819] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:13.082021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.092335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bf8fd-6364-49c3-a660-2f8733176c96 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.098130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.098838 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.099117 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.099388 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.100099 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:48:13.100398 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.102015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.114214 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1416/2820] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:48:13.118751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b2ab-f425-469a-b51e-596414260127 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.128758 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248b2ab-f425-469a-b51e-596414260127 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.133962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.134648 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.134928 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.135202 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.135869 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.137913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.158319 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:13.166814 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1405/2821] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:13.171127 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.181527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70127482-68d1-4032-bff8-176f817bd503 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.215867 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:13.246030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.247160 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.247443 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.247715 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.248406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.250332 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.270868 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1417/2822] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:13.275239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.285842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c739099-4376-49fc-9e27-e674ba76c0ce admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.290637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.291307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.291589 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.291863 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.292597 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.294471 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.305305 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1406/2823] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:13.309804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.320122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4b63d-2064-4b67-b5d1-9f7dc7cb5a4e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.324785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1be32c-b81f-419e-bcef-46869c4e48bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.325413 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1be32c-b81f-419e-bcef-46869c4e48bb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.325691 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1be32c-b81f-419e-bcef-46869c4e48bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.325961 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1be32c-b81f-419e-bcef-46869c4e48bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.326726 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1be32c-b81f-419e-bcef-46869c4e48bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.328359 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1be32c-b81f-419e-bcef-46869c4e48bb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.338462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1418/2824] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:48:13.342992 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.353098 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffc737-3a4c-45da-9f78-4595d383b493 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.358256 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.359197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.359481 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.359760 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] PATH_INFO: `/v3/projects/dd274762e76a4052862de3eda8128f31/users/414dda3766534d2a9125aa55a974c173/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.378126 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd274762e76a4052862de3eda8128f31, user_id=414dda3766534d2a9125aa55a974c173, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.380842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.396414 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1407/2825] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:13 2022] PUT /identity/v3/projects/dd274762e76a4052862de3eda8128f31/users/414dda3766534d2a9125aa55a974c173/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:13.407050 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.407357 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.407630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.435281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] MFA Rules not processed for user `414dda3766534d2a9125aa55a974c173`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:13.436504 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:13.503915 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1419/2826] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:13.508806 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.577152 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c3f1-b985-464e-a79f-2f999782697d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.582085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.582795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.583077 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.583349 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.584036 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.585935 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.624804 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1408/2827] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:13.629784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.640616 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567136db-1af7-4a8d-b9a4-01940ba948b1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.705604 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.706271 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.706595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.706873 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.707549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.709495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.730663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1420/2828] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:48:13.735549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.746980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cacdd7e-6ff4-44bd-9a8d-6be47db8465a admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.753154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.754207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.754706 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.755192 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.756372 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.758999 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.777293 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1409/2829] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:13.783721 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.798147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc6246-34f8-40f2-a40b-07df8e2ea4d2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.804058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.804949 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.805225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.805497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] PATH_INFO: `/v3/projects/1f98608402a14585aeccd1894db2ed9d/users/0abf93c638944602861f595480859f8b/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.822083 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f98608402a14585aeccd1894db2ed9d, user_id=0abf93c638944602861f595480859f8b, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.825047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.839221 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1421/2830] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:13 2022] PUT /identity/v3/projects/1f98608402a14585aeccd1894db2ed9d/users/0abf93c638944602861f595480859f8b/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:13.843442 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.854487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972d995-77b0-4604-9ca4-ca0ef2b5bfe0 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.914119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4f30-1954-4fc6-ae6a-617d9bb765ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.914825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4f30-1954-4fc6-ae6a-617d9bb765ca admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.915107 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4f30-1954-4fc6-ae6a-617d9bb765ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.915408 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4f30-1954-4fc6-ae6a-617d9bb765ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.916148 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4f30-1954-4fc6-ae6a-617d9bb765ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.917759 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a4f30-1954-4fc6-ae6a-617d9bb765ca admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.930281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1410/2831] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:13.935281 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:13.946996 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151fc1e-2e80-4620-8169-2392421ae3d2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:13.953075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:13.954639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.955116 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.955583 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] PATH_INFO: `/v3/projects/1f98608402a14585aeccd1894db2ed9d/users/0abf93c638944602861f595480859f8b/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:13.959238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f98608402a14585aeccd1894db2ed9d, user_id=0abf93c638944602861f595480859f8b, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:13.964166 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:13.968699 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:13.977743 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1422/2832] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:13 2022] PUT /identity/v3/projects/1f98608402a14585aeccd1894db2ed9d/users/0abf93c638944602861f595480859f8b/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:13.989772 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:13.990103 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:13.990411 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:14.025018 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] MFA Rules not processed for user `0abf93c638944602861f595480859f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:14.026257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:14.047846 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:14.102588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1411/2833] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:14.113901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:14.116125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:14.177192 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723b455-5b57-47da-917f-6e33f4d2a3a4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:14.181716 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05f8a5-2820-45b4-9227-2bcf4a9a233b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:14.182379 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f8a5-2820-45b4-9227-2bcf4a9a233b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:14.182697 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f8a5-2820-45b4-9227-2bcf4a9a233b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:14.182999 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05f8a5-2820-45b4-9227-2bcf4a9a233b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:14.193315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05f8a5-2820-45b4-9227-2bcf4a9a233b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:14.194819 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05f8a5-2820-45b4-9227-2bcf4a9a233b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:14.204871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1423/2834] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:14.589383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:14.697390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cdacc-2b41-4f44-b365-ab5a6e20f2f4 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:14.708996 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:14.710451 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:14.711096 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:14.711281 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] PATH_INFO: `/v3/users/414dda3766534d2a9125aa55a974c173` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:14.713431 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414dda3766534d2a9125aa55a974c173)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:14.716653 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:14.773207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414dda3766534d2a9125aa55a974c173'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:14.781520 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414dda3766534d2a9125aa55a974c173'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:14.782166 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414dda3766534d2a9125aa55a974c173'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:14.782795 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414dda3766534d2a9125aa55a974c173'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:14.789914 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414dda3766534d2a9125aa55a974c173'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:14.808864 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1412/2835] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:14 2022] DELETE /identity/v3/users/414dda3766534d2a9125aa55a974c173 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:14.813587 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:14.813852 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:14.814150 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:14.832064 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:14.833274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:14.897623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1424/2836] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:14.908099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:14.909763 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:14.959415 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:14.977495 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6709f1-0ec0-4fd9-b3f0-5bb550d1cdf9 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:14.981242 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:14.981910 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:14.982185 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:14.982459 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:14.993198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:14.994988 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:15.007179 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1413/2837] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:15.296990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:15.298599 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:15.315814 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53e4f9-6df4-430b-b50b-49ec36f36d8a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:15.323236 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:15.324941 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:15.325366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:15.325854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] PATH_INFO: `/v3/projects/dd274762e76a4052862de3eda8128f31` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:15.327857 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd274762e76a4052862de3eda8128f31)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:15.331476 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:15.399670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd274762e76a4052862de3eda8128f31', 'request_id': 'req-905e4256-ff06-41f7-99fa-10d7730aed30'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.413326 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd274762e76a4052862de3eda8128f31', 'request_id': 'req-905e4256-ff06-41f7-99fa-10d7730aed30'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.420926 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd274762e76a4052862de3eda8128f31', 'request_id': 'req-905e4256-ff06-41f7-99fa-10d7730aed30'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.432614 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] The token cache is being invalidate because project dd274762e76a4052862de3eda8128f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:15.432906 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.433639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.435291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1425/2838] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:15 2022] DELETE /identity/v3/projects/dd274762e76a4052862de3eda8128f31 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:15.439384 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:15.451997 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca823d7-bd2b-489c-92ea-f41cdc9a4fc4 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:15.515391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:15.516176 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:15.516457 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:15.516727 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] PATH_INFO: `/v3/users/0abf93c638944602861f595480859f8b` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:15.517934 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0abf93c638944602861f595480859f8b)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:15.519870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:15.571275 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0abf93c638944602861f595480859f8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.580090 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0abf93c638944602861f595480859f8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.580753 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0abf93c638944602861f595480859f8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.581395 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0abf93c638944602861f595480859f8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.588820 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-74f7bccd-246d-475d-840a-7794d8ef6a16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0abf93c638944602861f595480859f8b'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:15.605911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1414/2839] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:15 2022] DELETE /identity/v3/users/0abf93c638944602861f595480859f8b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:15.892354 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:15.894081 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:15.911314 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e4256-ff06-41f7-99fa-10d7730aed30 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:15.936860 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:15.980633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:15.981537 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:15.981842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:15.982112 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] PATH_INFO: `/v3/projects/1f98608402a14585aeccd1894db2ed9d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:15.983365 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f98608402a14585aeccd1894db2ed9d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:15.985291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.034449 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f98608402a14585aeccd1894db2ed9d', 'request_id': 'req-2393e488-cfdd-46c4-943c-e8dd0f082cab'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:16.042283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f98608402a14585aeccd1894db2ed9d', 'request_id': 'req-2393e488-cfdd-46c4-943c-e8dd0f082cab'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:16.049114 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f98608402a14585aeccd1894db2ed9d', 'request_id': 'req-2393e488-cfdd-46c4-943c-e8dd0f082cab'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:16.060310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] The token cache is being invalidate because project 1f98608402a14585aeccd1894db2ed9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:16.060605 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:16.061283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2393e488-cfdd-46c4-943c-e8dd0f082cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:16.062939 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1426/2840] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:15 2022] DELETE /identity/v3/projects/1f98608402a14585aeccd1894db2ed9d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:16.067956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.068499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.068499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.087681 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:16.088926 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:16.145332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:16.155154 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1415/2841] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:16.164255 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.164639 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.164973 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.186179 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:16.187450 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:16.196643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1427/2842] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:16.201496 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.212023 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadca7d0-e873-43c4-8497-daf7c641cc8a None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.217761 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.218428 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.218741 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.219016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.219754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:48:16.220060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.221674 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.234482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1416/2843] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:48:16.239198 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.249501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8435d1-311a-4c9c-8091-5cd768d0a77d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.255086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.255740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.256017 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.256287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.256978 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.258919 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.286665 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1428/2844] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:16.291088 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.305555 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b878484-afd7-42f6-a182-46b8a35a37ea admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.367744 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.368409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.368691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.368967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.369973 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.372043 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.400836 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1417/2845] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:16.405625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.416995 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5b058-959a-4646-b55c-b646ffbf7551 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.422274 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.425294 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.425595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.425879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.426663 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.428329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.440901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1429/2846] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:16.445224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.455874 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1894f9-8adc-4f54-b69a-7e38a828f19d admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.469810 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401bd1b-79cb-4240-9393-2822d6438248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.470508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401bd1b-79cb-4240-9393-2822d6438248 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.470828 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401bd1b-79cb-4240-9393-2822d6438248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.471104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401bd1b-79cb-4240-9393-2822d6438248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.471872 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401bd1b-79cb-4240-9393-2822d6438248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.473487 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401bd1b-79cb-4240-9393-2822d6438248 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.486856 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1418/2847] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:16.491351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.504917 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64391dc-f418-44f6-bd0c-ff06d96f1fd9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.510744 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.511638 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.511983 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.512245 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] PATH_INFO: `/v3/projects/24cf147cf4fd460a9e47d3dff0b26538/users/10cdbfc74fbe4564b26e1393f49b46a2/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.528630 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24cf147cf4fd460a9e47d3dff0b26538, user_id=10cdbfc74fbe4564b26e1393f49b46a2, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.531366 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.551352 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1430/2848] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:16 2022] PUT /identity/v3/projects/24cf147cf4fd460a9e47d3dff0b26538/users/10cdbfc74fbe4564b26e1393f49b46a2/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:16.563850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.564125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.564409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.593535 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] MFA Rules not processed for user `10cdbfc74fbe4564b26e1393f49b46a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:16.594799 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:16.668622 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1419/2849] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:16.673785 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.729341 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:16.744448 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668040d-2a24-4934-ba8d-7439dedac181 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.749818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.750475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.750784 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.751051 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.751735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.753633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.780782 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1431/2850] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:16.785079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.797257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6925b99-01d7-421b-b688-dc56aa0b2e01 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.851850 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:16.860140 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.860821 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.861255 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.861739 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.862446 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.864391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.886282 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1420/2851] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:16 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 15:48:16.890446 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.901137 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d6f43-151a-48e3-ae9a-9314a7b4b06b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.906466 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.907161 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.907452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.907719 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.908456 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.910463 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.924009 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1432/2852] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:16.928424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.939757 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9bd0-ca57-4b26-8dad-242dfc6d0a43 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.945263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687460b7-8786-4429-acbc-75b5b620ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.945953 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687460b7-8786-4429-acbc-75b5b620ac22 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.946240 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687460b7-8786-4429-acbc-75b5b620ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.946538 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687460b7-8786-4429-acbc-75b5b620ac22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:16.947280 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687460b7-8786-4429-acbc-75b5b620ac22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:16.948913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687460b7-8786-4429-acbc-75b5b620ac22 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:16.960618 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1421/2853] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:16.965061 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:16.976035 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9c478-95fc-497b-991d-9f697f1cfade admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:16.982085 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:16.983452 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:16.983776 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:16.984176 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] PATH_INFO: `/v3/projects/202c92916c674b0794d3a4e7244e7eed/users/8cb47547375f40c18de7efcc5267b7e4/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.000826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=202c92916c674b0794d3a4e7244e7eed, user_id=8cb47547375f40c18de7efcc5267b7e4, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.003845 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.020684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1433/2854] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:16 2022] PUT /identity/v3/projects/202c92916c674b0794d3a4e7244e7eed/users/8cb47547375f40c18de7efcc5267b7e4/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:17.032303 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.032573 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.032898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.061460 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] MFA Rules not processed for user `8cb47547375f40c18de7efcc5267b7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:17.062779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:17.128327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1422/2855] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:17.132992 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.198805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06e1b-782a-4f37-858d-930a1fdd5a81 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.204182 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.204849 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.205128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.205401 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.206124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.208223 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.240564 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1434/2856] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:17 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:17.241890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.253409 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbe19a-374c-4477-a460-bff7df95d63e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.313119 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.313800 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.314081 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.314353 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.315104 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.317034 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.337841 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1423/2857] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:17 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:17.342216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.352965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e14ff-f483-4822-bd63-58d2b641f7df admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.358460 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.359220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.359486 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.359821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.360570 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.362158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.374038 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1435/2858] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:17.378436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.389267 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92b07b-4bf2-4728-b8c8-0878b5a904b7 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.394902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.395834 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.396118 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.396393 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] PATH_INFO: `/v3/projects/d1a83e7a4aaf4831b4d7311bbc4785e6/users/b250feee0ab34650969f449b91e16e94/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.412690 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a83e7a4aaf4831b4d7311bbc4785e6, user_id=b250feee0ab34650969f449b91e16e94, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.415444 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.435517 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1424/2859] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:17 2022] PUT /identity/v3/projects/d1a83e7a4aaf4831b4d7311bbc4785e6/users/b250feee0ab34650969f449b91e16e94/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:17.439626 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.450243 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce5e037-47e5-46a2-baeb-e2b5fe700c2f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.512264 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0433605-e144-4143-ab72-2113d94a39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.512958 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433605-e144-4143-ab72-2113d94a39b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.513241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433605-e144-4143-ab72-2113d94a39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.513511 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0433605-e144-4143-ab72-2113d94a39b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.514275 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0433605-e144-4143-ab72-2113d94a39b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.515909 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0433605-e144-4143-ab72-2113d94a39b6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.526301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1436/2860] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:17.530792 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.541691 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fcbf01-6ad9-4302-8afc-22646cc7bd5b admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.547529 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.548467 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.548748 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.549371 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] PATH_INFO: `/v3/projects/d1a83e7a4aaf4831b4d7311bbc4785e6/users/b250feee0ab34650969f449b91e16e94/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.551631 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a83e7a4aaf4831b4d7311bbc4785e6, user_id=b250feee0ab34650969f449b91e16e94, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.554364 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.565771 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1425/2861] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:17 2022] PUT /identity/v3/projects/d1a83e7a4aaf4831b4d7311bbc4785e6/users/b250feee0ab34650969f449b91e16e94/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:17.576610 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.576879 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.577163 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.596848 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:17.608589 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] MFA Rules not processed for user `b250feee0ab34650969f449b91e16e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:17.609850 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:17.689884 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1437/2862] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:17.704177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:17.706259 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:17.731955 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:17.772497 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cca240-39b3-4964-92aa-c3cc2c2e16a6 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:17.776478 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:17.777122 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:17.777400 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:17.777668 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:17.835658 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:17.837105 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:17.846595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1426/2863] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:19.369956 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:19.371698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:19.393990 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2587c-6c2c-4110-8d5d-82775f97fad7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:19.397854 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:19.398551 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:19.398870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:19.399146 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:19.469238 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:19.470772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:19.480251 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1438/2864] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:20.057070 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:20.066252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396d5b-d8fc-417d-b248-1a694ebf04f5 service cinder] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:20.070215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6215da-8bd1-4d81-b8fb-5b64d30cc205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:20.071642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6215da-8bd1-4d81-b8fb-5b64d30cc205 service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:20.071642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6215da-8bd1-4d81-b8fb-5b64d30cc205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:20.071642 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6215da-8bd1-4d81-b8fb-5b64d30cc205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:20.081873 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6215da-8bd1-4d81-b8fb-5b64d30cc205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:20.083342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6215da-8bd1-4d81-b8fb-5b64d30cc205 service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:20.093199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1427/2865] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:24.173295 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:24.241030 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9153a97-dc20-41ff-beaf-94511a2e38e4 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:24.246826 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:24.247832 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:24.248122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:24.248393 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] PATH_INFO: `/v3/users/10cdbfc74fbe4564b26e1393f49b46a2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:24.249675 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=10cdbfc74fbe4564b26e1393f49b46a2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:24.251660 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:24.291445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:24.302346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10cdbfc74fbe4564b26e1393f49b46a2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.309694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10cdbfc74fbe4564b26e1393f49b46a2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.318733 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10cdbfc74fbe4564b26e1393f49b46a2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.319485 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10cdbfc74fbe4564b26e1393f49b46a2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.320234 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10cdbfc74fbe4564b26e1393f49b46a2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.339842 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1439/2866] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:24 2022] DELETE /identity/v3/users/10cdbfc74fbe4564b26e1393f49b46a2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:24.345085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:24.345613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:24.345613 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:24.364327 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:24.365548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:24.428406 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1428/2867] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:24.438918 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:24.440453 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:24.502296 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2aa54b-7c41-4192-932f-54fa53a6fd7d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:24.506825 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:24.507487 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:24.507775 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:24.508048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:24.519528 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:24.521001 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:24.531287 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1440/2868] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:24.816947 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:24.818502 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:24.831796 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf940a-94e5-45c2-8619-a880f5a4cfca None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:24.837207 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:24.838253 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:24.838694 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:24.839074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] PATH_INFO: `/v3/projects/24cf147cf4fd460a9e47d3dff0b26538` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:24.841239 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24cf147cf4fd460a9e47d3dff0b26538)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:24.843230 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:24.896679 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24cf147cf4fd460a9e47d3dff0b26538', 'request_id': 'req-b8304080-04c2-41cd-b301-7f76d585d416'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.898475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:24.903704 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24cf147cf4fd460a9e47d3dff0b26538', 'request_id': 'req-b8304080-04c2-41cd-b301-7f76d585d416'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.910198 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24cf147cf4fd460a9e47d3dff0b26538', 'request_id': 'req-b8304080-04c2-41cd-b301-7f76d585d416'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.924715 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] The token cache is being invalidate because project 24cf147cf4fd460a9e47d3dff0b26538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:24.925007 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.925629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:24.927279 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1429/2869] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:24 2022] DELETE /identity/v3/projects/24cf147cf4fd460a9e47d3dff0b26538 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:24.931437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:24.943712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24c34f3-a807-424e-bf03-86997ee33d4e service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:25.021717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:25.023194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:25.023694 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:25.024183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] PATH_INFO: `/v3/users/8cb47547375f40c18de7efcc5267b7e4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:25.026282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb47547375f40c18de7efcc5267b7e4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:25.030152 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:25.096805 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb47547375f40c18de7efcc5267b7e4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.103815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb47547375f40c18de7efcc5267b7e4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.105492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:25.113195 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb47547375f40c18de7efcc5267b7e4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.114005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb47547375f40c18de7efcc5267b7e4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.114946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb47547375f40c18de7efcc5267b7e4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.134301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1441/2870] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:24 2022] DELETE /identity/v3/users/8cb47547375f40c18de7efcc5267b7e4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:25.415190 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:25.417600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:25.432486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8304080-04c2-41cd-b301-7f76d585d416 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:25.493086 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:25.493976 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:25.494264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:25.494574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] PATH_INFO: `/v3/projects/202c92916c674b0794d3a4e7244e7eed` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:25.496121 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=202c92916c674b0794d3a4e7244e7eed)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:25.498112 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:25.549430 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '202c92916c674b0794d3a4e7244e7eed', 'request_id': 'req-959b9b64-f7bc-41fa-a7ae-687f613edf38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.556082 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '202c92916c674b0794d3a4e7244e7eed', 'request_id': 'req-959b9b64-f7bc-41fa-a7ae-687f613edf38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.562277 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '202c92916c674b0794d3a4e7244e7eed', 'request_id': 'req-959b9b64-f7bc-41fa-a7ae-687f613edf38'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.575881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] The token cache is being invalidate because project 202c92916c674b0794d3a4e7244e7eed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:25.576173 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.576825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.578381 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1430/2871] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:25 2022] DELETE /identity/v3/projects/202c92916c674b0794d3a4e7244e7eed => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:25.582641 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:25.584075 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:25.597451 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0be847-6b98-4600-8b7f-0c4fa0584a95 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:25.658745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:25.659633 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:25.659911 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:25.660181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] PATH_INFO: `/v3/users/b250feee0ab34650969f449b91e16e94` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:25.661405 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b250feee0ab34650969f449b91e16e94)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:25.663403 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:25.710541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b250feee0ab34650969f449b91e16e94'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.717434 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b250feee0ab34650969f449b91e16e94'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.725580 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b250feee0ab34650969f449b91e16e94'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.726187 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b250feee0ab34650969f449b91e16e94'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.726811 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3fb36f22-dae1-4347-bc6c-62fa226ee81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b250feee0ab34650969f449b91e16e94'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:25.745477 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1442/2872] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:48:25 2022] DELETE /identity/v3/users/b250feee0ab34650969f449b91e16e94 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:26.026252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.027842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:26.047247 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b9b64-f7bc-41fa-a7ae-687f613edf38 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.104969 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.105747 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.106080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.106334 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] PATH_INFO: `/v3/projects/d1a83e7a4aaf4831b4d7311bbc4785e6` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.107600 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a83e7a4aaf4831b4d7311bbc4785e6)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.109514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.122130 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:26.157684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a83e7a4aaf4831b4d7311bbc4785e6', 'request_id': 'req-76d4e502-8e59-4e7d-9367-0934a1e4bff8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:26.163781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a83e7a4aaf4831b4d7311bbc4785e6', 'request_id': 'req-76d4e502-8e59-4e7d-9367-0934a1e4bff8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:26.169494 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a83e7a4aaf4831b4d7311bbc4785e6', 'request_id': 'req-76d4e502-8e59-4e7d-9367-0934a1e4bff8'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:26.183200 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] The token cache is being invalidate because project d1a83e7a4aaf4831b4d7311bbc4785e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:48:26.183485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:26.184203 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-76d4e502-8e59-4e7d-9367-0934a1e4bff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:48:26.185783 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1431/2873] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:48:26 2022] DELETE /identity/v3/projects/d1a83e7a4aaf4831b4d7311bbc4785e6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:26.190846 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.191132 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.191407 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.211168 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:26.212406 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:26.265683 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:26.279828 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1443/2874] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:26.288791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.289093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.289424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.307962 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:26.309379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:26.317990 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1432/2875] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:26.322437 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.332661 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631edce0-95a1-4cca-9bf2-0121c8b12944 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.337458 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.338122 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.338400 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.338701 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.339431 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:48:26.339731 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.341552 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.353608 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1444/2876] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 15:48:26.358217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.367678 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241520e-8355-4d03-83e2-05c3f94cdd3e None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.373212 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.373881 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.374159 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.374434 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.375188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.377113 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.405449 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1433/2877] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:48:26 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:26.409693 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.420490 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f9f7d-ec15-43f8-969d-f1d471bf5ab5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.479600 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.480269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.480549 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.480817 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.481813 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.483790 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.506455 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1445/2878] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:48:26 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:48:26.510418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.522134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf33048d-a3e8-4d15-9fc2-09d498538e9f admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.527652 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.528528 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.528740 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.529016 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.529749 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.531426 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.542244 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1434/2879] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:26.546445 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.556861 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1656f0e-8d77-432c-86db-eac51ebf2221 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.562505 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f4293-d1e2-4648-84b0-d1e078c01966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.563235 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4293-d1e2-4648-84b0-d1e078c01966 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.563513 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4293-d1e2-4648-84b0-d1e078c01966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.563786 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f4293-d1e2-4648-84b0-d1e078c01966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.564518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f4293-d1e2-4648-84b0-d1e078c01966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.566130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f4293-d1e2-4648-84b0-d1e078c01966 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.576670 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1446/2880] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:48:26.581078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.591698 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798fa637-8554-4074-95db-ea8b60cfc4fc admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.597078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.597967 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.598252 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.598541 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] PATH_INFO: `/v3/projects/187f2cb0f64542ec933321f23123809c/users/8ab8bff73d53469aa1d2a38650b6b310/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.614932 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=187f2cb0f64542ec933321f23123809c, user_id=8ab8bff73d53469aa1d2a38650b6b310, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.617552 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.632514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1435/2881] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:48:26 2022] PUT /identity/v3/projects/187f2cb0f64542ec933321f23123809c/users/8ab8bff73d53469aa1d2a38650b6b310/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:48:26.643194 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.643734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.643734 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.671762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] MFA Rules not processed for user `8ab8bff73d53469aa1d2a38650b6b310`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:48:26.673028 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:26.738340 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1447/2882] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:48:26.750346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:48:26.752229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:48:26.812606 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4a95-75cd-4d63-8f17-2843bc230109 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:48:26.816722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee176ffb-cb31-462d-b8d7-ec4a0222ce8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4cfd0067b59048f1850e8eb6e94640ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:48:26.817383 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee176ffb-cb31-462d-b8d7-ec4a0222ce8d service cinder] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:48:26.817667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee176ffb-cb31-462d-b8d7-ec4a0222ce8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:48:26.817945 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee176ffb-cb31-462d-b8d7-ec4a0222ce8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:48:26.827014 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee176ffb-cb31-462d-b8d7-ec4a0222ce8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:48:26.828462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee176ffb-cb31-462d-b8d7-ec4a0222ce8d service cinder] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:48:26.837805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1436/2883] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:48:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:48:36.735555 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:48:36.833332 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:15.834692 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:15.911093 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00db91a-26e7-40c7-8956-5c12801a8d4a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:15.916643 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:15.917527 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:15.917802 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:15.918068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] PATH_INFO: `/v3/users/8ab8bff73d53469aa1d2a38650b6b310` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:15.919374 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab8bff73d53469aa1d2a38650b6b310)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:15.921260 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:15.969897 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab8bff73d53469aa1d2a38650b6b310'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:15.977125 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab8bff73d53469aa1d2a38650b6b310'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:15.990386 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab8bff73d53469aa1d2a38650b6b310'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:15.992325 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab8bff73d53469aa1d2a38650b6b310'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:15.993609 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab8bff73d53469aa1d2a38650b6b310'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:16.017433 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1448/2884] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:49:15 2022] DELETE /identity/v3/users/8ab8bff73d53469aa1d2a38650b6b310 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:16.022146 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.022689 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.022805 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.040776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:16.041987 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:16.105462 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1437/2885] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:16.115636 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:16.117089 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:16.181129 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bd55a-2998-42c5-92d9-2ecee26719a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:16.186058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a0d07-8ffe-4adb-ad12-cb072c3e4138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:16.186740 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0d07-8ffe-4adb-ad12-cb072c3e4138 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.187019 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0d07-8ffe-4adb-ad12-cb072c3e4138 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.187269 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a0d07-8ffe-4adb-ad12-cb072c3e4138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.197207 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a0d07-8ffe-4adb-ad12-cb072c3e4138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:16.198678 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a0d07-8ffe-4adb-ad12-cb072c3e4138 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:16.208735 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1449/2886] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:16.511625 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:16.513288 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:16.525879 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971080e6-6e33-48c5-a425-6066255579e1 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:16.531137 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:16.532030 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.532306 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.533567 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] PATH_INFO: `/v3/projects/187f2cb0f64542ec933321f23123809c` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.535174 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=187f2cb0f64542ec933321f23123809c)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:16.537085 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:16.585980 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '187f2cb0f64542ec933321f23123809c', 'request_id': 'req-c7b34eed-23ad-4782-84a1-ce51201e4fed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:16.592699 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '187f2cb0f64542ec933321f23123809c', 'request_id': 'req-c7b34eed-23ad-4782-84a1-ce51201e4fed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:16.598607 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '187f2cb0f64542ec933321f23123809c', 'request_id': 'req-c7b34eed-23ad-4782-84a1-ce51201e4fed'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:16.612298 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] The token cache is being invalidate because project 187f2cb0f64542ec933321f23123809c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:49:16.612514 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:16.613170 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-c7b34eed-23ad-4782-84a1-ce51201e4fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:49:16.614807 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1438/2887] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:49:16 2022] DELETE /identity/v3/projects/187f2cb0f64542ec933321f23123809c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:16.621188 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.621681 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.622021 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.640930 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:16.642140 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:16.660046 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:16.704756 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1450/2888] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:16.713617 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.713904 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.714177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.734080 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:16.735313 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:16.744177 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1439/2889] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:16.749471 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:16.765935 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412984d7-e87e-49da-bf4d-69f04a46f278 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:16.771546 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:16.772212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.772497 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.772772 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.773507 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 15:49:16.773812 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:16.775520 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:16.788981 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1451/2890] 10.222.0.40 () {58 vars in 1152 bytes} [Fri Jul 15 15:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 15:49:16.793391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:16.803199 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8093000-5ead-43c3-9b63-8a621b76a666 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:16.808733 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:16.809398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.809667 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.810057 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.811307 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:16.813473 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:16.845776 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1440/2891] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:49:16 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:49:16.850068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:16.860866 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11104d25-b91d-4a50-9a2f-b84e55cadb88 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:16.922920 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:16.923516 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.923795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.924071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.924762 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:16.926818 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:16.954809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1452/2892] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:49:16 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:49:16.959568 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:16.971224 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9df60-6f12-4000-94bd-87cf80c968c5 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:16.978684 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:16.979553 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:16.979930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:16.980272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:16.981217 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:16.983024 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:16.994093 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1441/2893] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 15:49:16.999847 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:17.010454 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03e76c-35d4-4733-8b7c-c91e3d4d24f9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:17.015688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fce43d-43da-4058-b7ef-2a9d5c6f38db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:17.016351 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fce43d-43da-4058-b7ef-2a9d5c6f38db admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:17.016627 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fce43d-43da-4058-b7ef-2a9d5c6f38db admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:17.016896 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fce43d-43da-4058-b7ef-2a9d5c6f38db admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:17.017622 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fce43d-43da-4058-b7ef-2a9d5c6f38db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:17.019321 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fce43d-43da-4058-b7ef-2a9d5c6f38db admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:17.033026 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1453/2894] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:49:17.037508 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:17.048355 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233ccdf-3f6c-42bb-aa2e-5f89a480c282 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:17.053993 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:17.054930 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:17.055211 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:17.055485 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] PATH_INFO: `/v3/projects/b031042c48094fdcba11a1a647323b94/users/af392d5574cd41399105a3ef96f6d0cf/roles/c175ea449c794ef699b7b13a0e6803d5` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:17.072338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b031042c48094fdcba11a1a647323b94, user_id=af392d5574cd41399105a3ef96f6d0cf, role_id=c175ea449c794ef699b7b13a0e6803d5)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:17.075048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:17.088929 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1442/2895] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:49:17 2022] PUT /identity/v3/projects/b031042c48094fdcba11a1a647323b94/users/af392d5574cd41399105a3ef96f6d0cf/roles/c175ea449c794ef699b7b13a0e6803d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:17.095135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423fad4-9f40-43aa-b0b3-e30126873362 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:17.095481 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423fad4-9f40-43aa-b0b3-e30126873362 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:17.095795 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423fad4-9f40-43aa-b0b3-e30126873362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:17.116350 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0423fad4-9f40-43aa-b0b3-e30126873362 None None] MFA Rules not processed for user `1b155a91a9264755b87c158c22553253`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:17.117623 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0423fad4-9f40-43aa-b0b3-e30126873362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:17.184000 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1454/2896] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:17.197333 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:17.200311 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:17.232342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:17.285870 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdda7e6-3ac3-4554-99ea-b0f037863cdd admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:17.289751 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:17.290461 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:17.290778 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:17.291052 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:17.301272 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:17.302994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:17.314591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1443/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:21.588791 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.589313 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.589447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.620625 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] MFA Rules not processed for user `af392d5574cd41399105a3ef96f6d0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:21.622492 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:21.643022 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:21.702143 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1455/2898] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:21.706683 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.717164 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ad38-717a-46c1-8bbe-25ff846bdb49 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:21.722163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:21.722959 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.723345 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.723697 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] PATH_INFO: `/v3/projects` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.724524 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:21.726574 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:21.758707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1444/2899] 10.222.0.40 () {60 vars in 1152 bytes} [Fri Jul 15 15:49:21 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:49:21.759946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.771685 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca92101-8528-4193-a494-c7c14f6223dc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:21.832344 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:21.833037 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.833318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.833585 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.834292 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:21.836335 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:21.857878 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1456/2900] 10.222.0.40 () {60 vars in 1143 bytes} [Fri Jul 15 15:49:21 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 15:49:21.861956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.873707 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859c7c5-7137-4bf0-a1f9-2105546af2db admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:21.878512 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:21.879233 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.879550 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.879829 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.880554 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:21.882169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:21.895342 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1445/2901] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:49:21.900417 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.915901 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e874-799e-435e-8dad-305928935a2a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:21.923068 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:21.924475 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:21.924889 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:21.925307 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] PATH_INFO: `/v3/projects/d6c186e135c84cb6aa01006cd04de49c/users/65443b43f79a4024901404f72e09a01f/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:21.947333 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c186e135c84cb6aa01006cd04de49c, user_id=65443b43f79a4024901404f72e09a01f, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:21.950149 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:21.966184 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1457/2902] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:49:21 2022] PUT /identity/v3/projects/d6c186e135c84cb6aa01006cd04de49c/users/65443b43f79a4024901404f72e09a01f/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:21.971162 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:21.982008 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf86185-df5c-4e46-a3b1-5d81584e3d95 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:22.046360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0187082-c563-4b3b-b5e9-25fdaece23e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:22.047271 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187082-c563-4b3b-b5e9-25fdaece23e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:22.047591 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187082-c563-4b3b-b5e9-25fdaece23e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:22.047868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0187082-c563-4b3b-b5e9-25fdaece23e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:22.048661 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0187082-c563-4b3b-b5e9-25fdaece23e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:22.050548 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0187082-c563-4b3b-b5e9-25fdaece23e2 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:22.061754 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1446/2903] 10.222.0.40 () {58 vars in 1121 bytes} [Fri Jul 15 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 15:49:22.066252 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:22.077586 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c05616-954a-4b10-bccc-cbc5fc7e590a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:22.083177 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:22.084070 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:22.084346 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:22.084620 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] PATH_INFO: `/v3/projects/d6c186e135c84cb6aa01006cd04de49c/users/65443b43f79a4024901404f72e09a01f/roles/fb625882070a4b2bb3d8e14f8f69c4d5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:22.086809 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c186e135c84cb6aa01006cd04de49c, user_id=65443b43f79a4024901404f72e09a01f, role_id=fb625882070a4b2bb3d8e14f8f69c4d5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:22.089515 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:22.101225 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1458/2904] 10.222.0.40 () {60 vars in 1482 bytes} [Fri Jul 15 15:49:22 2022] PUT /identity/v3/projects/d6c186e135c84cb6aa01006cd04de49c/users/65443b43f79a4024901404f72e09a01f/roles/fb625882070a4b2bb3d8e14f8f69c4d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:49:26.904890 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e04c5-46da-45b0-9083-8461b75ba7d1 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:26.906134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e04c5-46da-45b0-9083-8461b75ba7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:26.906794 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e04c5-46da-45b0-9083-8461b75ba7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:26.958378 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-692e04c5-46da-45b0-9083-8461b75ba7d1 None None] MFA Rules not processed for user `65443b43f79a4024901404f72e09a01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:26.959827 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-692e04c5-46da-45b0-9083-8461b75ba7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:27.066911 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1447/2905] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:27.088186 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:27.090338 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:27.186885 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967544a-26c2-41be-9ca5-1b698f82a6a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:27.196130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:27.197331 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:27.197965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:27.198508 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:27.209301 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:27.280965 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:27.282800 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:27.293929 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1459/2906] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:28.409390 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:28.409938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:28.409938 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:28.429377 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] MFA Rules not processed for user `af392d5574cd41399105a3ef96f6d0cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:49:28.430716 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:28.438543 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1448/2907] 10.222.0.40 () {58 vars in 957 bytes} [Fri Jul 15 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:49:32.023548 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:32.025571 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:32.104036 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122e420-a6d7-4333-a8e0-2453cc9d4f41 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:32.109711 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:32.110535 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:32.110870 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:32.111319 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:32.113058 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:32.126668 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:32.128283 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:32.139212 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1460/2908] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:38.423501 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:42.134472 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=56 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:49:50.338309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:50.340004 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:50.403486 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c133f-bd2b-4e3f-a82f-435db3e3d820 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:50.407264 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:50.407913 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:50.408246 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:50.408521 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:50.410314 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:50.420722 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:50.422163 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:50.431089 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1449/2909] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:51.924291 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:51.925912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:51.940181 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2eca17-c3f1-4fd3-8917-7a3da5f05629 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:51.946928 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:51.947595 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:51.947875 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:51.948155 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:51.950199 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:51.963315 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:51.964937 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:51.977946 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1461/2910] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:49:57.445791 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:49:57.481309 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea7dd7-d388-4e3e-91b4-1f1504cb3d91 service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:49:57.491359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:49:57.492091 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:49:57.492380 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:49:57.492656 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:49:57.494729 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:49:57.511289 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:49:57.512902 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:49:57.527398 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1450/2911] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:49:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:50:01.971561 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:50:07.522028 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=32 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:50:08.045447 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:50:08.059816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9fba3-aac4-4905-9c36-90ad5c24da08 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:50:08.066572 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:50:08.067241 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:50:08.067518 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:50:08.067787 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:50:08.070259 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:50:08.084597 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:50:08.086220 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:50:08.098084 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1462/2912] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:50:11.101042 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:50:11.112586 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56478fa8-31a2-4c8b-bab2-270625c99c98 service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:50:11.117186 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:50:11.117848 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:50:11.118123 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:50:11.118391 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:50:11.130588 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:50:11.132078 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:50:11.143842 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1451/2913] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:50:18.093666 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:50:21.138755 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:52:34.550144 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:52:34.560541 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ecae85-fdf3-468c-ac4b-2dd85ea2c5d1 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:52:34.565124 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:52:34.565792 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:34.566071 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:34.566343 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:34.568816 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:52:34.579209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:52:34.580655 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:52:34.588964 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1463/2914] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:52:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:52:40.177595 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:52:40.186896 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730b7d7-5fe6-45d7-acf9-88336c6785ea service neutron] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:52:40.191417 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:52:40.192151 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:40.192611 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:40.192894 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:40.195079 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:52:40.209359 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:52:40.210824 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d99d8c-a045-4f39-a19a-3ba530327e81 service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:52:40.222379 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1452/2915] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:52:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:52:43.763329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:52:43.772821 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451015e-4f49-43e2-aa7e-dcb33181db0d service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:52:43.776934 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:52:43.777574 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:43.777859 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:43.778147 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:43.780128 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:52:43.794329 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:52:43.795815 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4656a1f4-852f-47ad-aba0-2cf5c5bd2421 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:52:43.807427 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1464/2916] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:52:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:52:50.217197 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=6 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:52:53.802096 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=12 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:52:54.492125 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb6b8-6401-41b2-bc66-59c9d84e5739 None None] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:54.492781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb6b8-6401-41b2-bc66-59c9d84e5739 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:54.492781 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb6b8-6401-41b2-bc66-59c9d84e5739 None None] PATH_INFO: `/` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:54.494075 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1453/2917] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:52:54.500712 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddf20b-6891-4be1-ba00-42e4ec141843 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:54.501086 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddf20b-6891-4be1-ba00-42e4ec141843 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:54.501421 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efddf20b-6891-4be1-ba00-42e4ec141843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:54.553047 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-efddf20b-6891-4be1-ba00-42e4ec141843 None None] MFA Rules not processed for user `0d9d155ea97f422490871907f4018532`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:52:54.555183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-efddf20b-6891-4be1-ba00-42e4ec141843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:52:54.562156 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1465/2918] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:52:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:52:54.576562 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb6b8-6401-41b2-bc66-59c9d84e5739 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:52:54.589175 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb6b8-6401-41b2-bc66-59c9d84e5739 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:52:54.593640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7dea0-b5cb-4dd7-808e-c57c35ba0cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:52:54.594295 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7dea0-b5cb-4dd7-808e-c57c35ba0cac service neutron] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:54.594608 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7dea0-b5cb-4dd7-808e-c57c35ba0cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:54.595422 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7dea0-b5cb-4dd7-808e-c57c35ba0cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:54.603974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7dea0-b5cb-4dd7-808e-c57c35ba0cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:52:54.605424 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7dea0-b5cb-4dd7-808e-c57c35ba0cac service neutron] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:52:54.614576 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1454/2919] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:52:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:52:54.713282 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22777e-a647-484a-87cc-b4d11b49635d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:54.713778 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22777e-a647-484a-87cc-b4d11b49635d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:54.714066 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22777e-a647-484a-87cc-b4d11b49635d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:54.715844 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1466/2920] 10.222.0.40 () {54 vars in 921 bytes} [Fri Jul 15 15:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 15:52:54.721812 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] REQUEST_METHOD: `POST` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:54.722100 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:54.722421 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:54.766065 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.auth.core [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] MFA Rules not processed for user `063c589447364966b64fcc2da5c1aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 15:52:54.767360 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:52:54.779961 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1455/2921] 10.222.0.40 () {58 vars in 1018 bytes} [Fri Jul 15 15:52:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 15:52:54.790966 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22777e-a647-484a-87cc-b4d11b49635d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:52:54.804183 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22777e-a647-484a-87cc-b4d11b49635d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:52:54.808684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:52:54.809370 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:52:54.809654 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:52:54.809924 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:52:54.822532 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:52:54.824015 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:52:54.835910 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1467/2922] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:52:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:53:04.762139 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=15 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:53:04.830539 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=19 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:34.812825 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:34.858868 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c557de7-605c-41dd-aad4-90f138814651 None None] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:34.864493 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:34.865142 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:34.865429 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:34.865710 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:34.876031 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:34.877623 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:34.892263 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1456/2923] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:44.883919 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=16 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:54:50.062480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:50.080950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50af8c-0539-43dc-8d56-8e0b930dd089 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:50.085536 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:50.086203 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:50.086480 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:50.086799 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:50.096804 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:50.098256 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:50.107777 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1468/2924] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:51.012530 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:51.026629 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402dbd6-dd9b-4d89-a1a3-179230ccf977 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:51.034445 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '063c589447364966b64fcc2da5c1aa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:51.035410 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:51.035888 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:51.036346 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:51.038436 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:51.063315 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:51.065188 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:51.077439 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1457/2925] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:54.992249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:55.001399 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c650-e688-4747-ad0b-44f3230bd76e service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:55.005135 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d9d155ea97f422490871907f4018532', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:55.005767 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:55.006048 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:55.006318 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:55.067950 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:55.070158 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:55.083824 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1469/2926] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:54:58.264784 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:54:58.277169 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64dd8f-1580-40b9-9bf0-868c0939d7ff service nova] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:54:58.282994 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:54:58.283676 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:54:58.283956 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:54:58.284229 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:54:58.286336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:54:58.319143 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:54:58.321160 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:54:58.340273 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1458/2927] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:54:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:01.336698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.355308 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a194b-2bdd-45db-b733-c1691e32a5ba service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.367871 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.369057 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.371136 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.371874 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.375912 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:01.395726 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.401232 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.403573 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.417499 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eabae7-9ebb-4ad3-bee6-fce8944d1d47 service placement] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.420677 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1470/2928] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:55:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:55:01.431549 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.433338 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.433887 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.434510 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] PATH_INFO: `/v3/users/af392d5574cd41399105a3ef96f6d0cf` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.436134 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af392d5574cd41399105a3ef96f6d0cf)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.439637 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.501822 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af392d5574cd41399105a3ef96f6d0cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.511423 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af392d5574cd41399105a3ef96f6d0cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.512048 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af392d5574cd41399105a3ef96f6d0cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.512738 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af392d5574cd41399105a3ef96f6d0cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.521974 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af392d5574cd41399105a3ef96f6d0cf'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.545918 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1459/2929] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:01 2022] DELETE /identity/v3/users/af392d5574cd41399105a3ef96f6d0cf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 15:55:01.738249 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.749227 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec69f-75d3-467d-b3b9-1a7b9107828e service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.810687 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.811440 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.811717 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.811994 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] PATH_INFO: `/v3/projects/b031042c48094fdcba11a1a647323b94` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.813729 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b031042c48094fdcba11a1a647323b94)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.815688 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:01.867611 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b031042c48094fdcba11a1a647323b94', 'request_id': 'req-cc5749d9-4616-4054-9433-7ee571460330'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.877612 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b031042c48094fdcba11a1a647323b94', 'request_id': 'req-cc5749d9-4616-4054-9433-7ee571460330'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.884100 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b031042c48094fdcba11a1a647323b94', 'request_id': 'req-cc5749d9-4616-4054-9433-7ee571460330'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.886172 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:01.897216 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] The token cache is being invalidate because project b031042c48094fdcba11a1a647323b94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:01.897501 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.898130 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:01.899771 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1471/2930] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:01 2022] DELETE /identity/v3/projects/b031042c48094fdcba11a1a647323b94 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:01.903898 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:01.905336 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:01.919193 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83fe10-e466-413e-8129-3a27b26bfd83 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:01.978640 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:01.979525 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:01.979803 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:01.980074 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] PATH_INFO: `/v3/users/65443b43f79a4024901404f72e09a01f` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:01.981331 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65443b43f79a4024901404f72e09a01f)` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:01.983285 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.024132 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:02.045099 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65443b43f79a4024901404f72e09a01f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.054257 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65443b43f79a4024901404f72e09a01f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.054944 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65443b43f79a4024901404f72e09a01f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.055675 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65443b43f79a4024901404f72e09a01f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.063482 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.notifications [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65443b43f79a4024901404f72e09a01f'} {{(pid=80270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.081215 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1460/2931] 10.222.0.40 () {58 vars in 1223 bytes} [Fri Jul 15 15:55:01 2022] DELETE /identity/v3/users/65443b43f79a4024901404f72e09a01f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:02.280603 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:55:02.282209 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:55:02.295754 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5749d9-4616-4054-9433-7ee571460330 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:55:02.358684 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b155a91a9264755b87c158c22553253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7405d44ad14342729b633b56109fe767', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:55:02.359698 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:55:02.360005 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:55:02.360280 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] PATH_INFO: `/v3/projects/d6c186e135c84cb6aa01006cd04de49c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:55:02.361495 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c186e135c84cb6aa01006cd04de49c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:55:02.363489 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:55:02.411237 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c186e135c84cb6aa01006cd04de49c', 'request_id': 'req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.419310 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c186e135c84cb6aa01006cd04de49c', 'request_id': 'req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.425931 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c186e135c84cb6aa01006cd04de49c', 'request_id': 'req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.436462 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] The token cache is being invalidate because project d6c186e135c84cb6aa01006cd04de49c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 15:55:02.436745 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.437441 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4b242c66-d9c8-47e2-ba0b-e7d7558567f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 15:55:02.439045 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1472/2932] 10.222.0.40 () {58 vars in 1232 bytes} [Fri Jul 15 15:55:02 2022] DELETE /identity/v3/projects/d6c186e135c84cb6aa01006cd04de49c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 15:55:12.077475 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=19,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:55:12.433673 dv-cinder-850015-1-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=54,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 15:59:39.802504 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Authenticating user token {{(pid=80270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 15:59:39.805084 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:39.915005 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e6776-18b0-4c6a-9158-a822c0bdba21 admin admin] Validating token access rules against request {{(pid=80270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 15:59:39.980060 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13549f6dc97241f28501b4c3e3c25959', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '485b3fa9c86c4ef6be14140d774f5730', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 15:59:39.980779 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] REQUEST_METHOD: `GET` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 15:59:39.981136 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 15:59:39.981418 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 15:59:39.983854 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.fernet_utils [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 15:59:39.996067 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 15:59:39.997662 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c32c7-1d12-4b49-b077-2304f3f745d4 service placement] RBAC: Authorization granted {{(pid=80270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 15:59:40.007912 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: [pid: 80270|app: 0|req: 1461/2933] 10.222.0.40 () {60 vars in 1365 bytes} [Fri Jul 15 15:59:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 15:59:50.003518 dv-cinder-850015-1-iscsi devstack@keystone.service[80270]: DEBUG dbcounter [-] [80270] Writing DB stats keystone:SELECT=69 {{(pid=80270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}